-- Logs begin at Mon 2022-08-08 17:10:30 UTC, end at Mon 2022-08-08 18:29:12 UTC. -- Aug 08 17:49:37.457244 dv-cinder-852364-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 17:49:37.468967 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 17:49:37 2022] *** Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: nodename: dv-cinder-852364-1-iscsi Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: machine: x86_64 Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: clock source: unix Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: pcre jit disabled Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: detected number of CPU cores: 4 Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: current working directory: / Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: detected binary path: /usr/bin/uwsgi-core Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: your processes number limit is 47680 Aug 08 17:49:37.470974 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: your memory page size is 4096 bytes Aug 08 17:49:37.472207 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: detected max file descriptor number: 2048 Aug 08 17:49:37.472207 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: lock engine: pthread robust mutexes Aug 08 17:49:37.472207 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: thunder lock: enabled Aug 08 17:49:37.472207 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 17:49:37.472207 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 17:49:37.511387 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: Python main interpreter initialized at 0x55beb62509c0 Aug 08 17:49:37.511387 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: python threads support enabled Aug 08 17:49:37.511387 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: your server socket listen backlog is limited to 100 connections Aug 08 17:49:37.511387 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: your mercy for graceful operations on workers is 90 seconds Aug 08 17:49:37.511730 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 17:49:37.511807 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: *** Operational MODE: preforking *** Aug 08 17:49:37.512054 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: *** uWSGI is running in multiple interpreter mode *** Aug 08 17:49:37.512054 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: spawned uWSGI master process (pid: 80389) Aug 08 17:49:37.512256 dv-cinder-852364-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 17:49:37.514350 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: spawned uWSGI worker 1 (pid: 80401, cores: 1) Aug 08 17:49:37.514350 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: spawned uWSGI worker 2 (pid: 80402, cores: 1) Aug 08 17:49:37.514350 dv-cinder-852364-1-iscsi devstack@keystone.service[80389]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 17:49:43.061397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 17:49:43.061397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: from cryptography.utils import int_from_bytes Aug 08 17:49:43.062140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 17:49:43.062140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: from cryptography.utils import int_from_bytes Aug 08 17:49:43.062552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 17:49:43.062552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: from cryptography.utils import int_from_bytes Aug 08 17:49:43.062660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 17:49:43.062660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: from cryptography.utils import int_from_bytes Aug 08 17:49:43.111170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 17:49:43.111366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 17:49:43.111450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 17:49:43.111541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] command line args: [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 17:49:43.111650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 17:49:43.111829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 17:49:43.111908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] command line args: [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 17:49:43.112009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ================================================================================ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 17:49:43.112126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 17:49:43.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ================================================================================ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 17:49:43.112391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] admin_token = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.112600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] admin_token = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.112693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] config_dir = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.112852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] config_dir = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.112929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.113041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.113127 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] config_source = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.113224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] config_source = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.113303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] debug = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.113405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] debug = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.113490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.113775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.113904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_config_append = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.114975 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.115079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.115181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_config_append = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.115283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_dir = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.115391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.115487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_file = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.115592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_dir = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.115726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.115804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_file = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.115927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.116009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.116131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.116217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.116296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.116445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.116553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.116684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.116803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.116885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.117009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.117117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.117202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.117352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.117442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.117587 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.117665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.117819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.117945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.118053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.118198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.118299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.118482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.118570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.118767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.118852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.119063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.119289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.119374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.119456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] publish_errors = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.119693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.119776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.119924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.120102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.120275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.120405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.120480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] publish_errors = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.120592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.120772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.120858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] standard_threads = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_journal = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_json = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.121953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_stderr = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.122100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] standard_threads = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.122179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] use_syslog = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.122345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.122421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.122601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.122679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.122842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.122927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.123044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_journal = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.123121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.123247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_json = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.123384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.123541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_stderr = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.123616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.123687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] use_syslog = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.123832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.123915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 17:49:43.124027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.124206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.124292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.124518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.124599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.124776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.124874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.125028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.125183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.125267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.125497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.125574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.125737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.125827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.125986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.126164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.126241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.126464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.126593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.126719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.126947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.127036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.127195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.127279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.127944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.128025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.128177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.128253 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.128423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.128518 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.128682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.128762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.128926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.129003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.129170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.129255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.129406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.129493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.129643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.129720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.129884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.129969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.130111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.130189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.130345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.130423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.130594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.130676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.130824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.130902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.131075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.131159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.131323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.131399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.131554 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.131638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.131784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.131861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.132013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.132096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.132247 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.132325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.132483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.132613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.132748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.132826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.132975 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.133059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.133204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.133281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.133436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.133538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.133675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.133780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.133905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.134007 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.134136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.134246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.134354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.134440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.134586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.134664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.134814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.134942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.135043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.135123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.135270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.135358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.135496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.135629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.135717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.135796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.135944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.136030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.136179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.136256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.136403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.136493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.136657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.136779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.136884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.136964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.137116 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.137208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.137347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.137468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.137585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.137663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.137810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.137891 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.138043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.138172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.138274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.138353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.138505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.138596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.138736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.138815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.138967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.139053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.139197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.139288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.139440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.139564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.139672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.139751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.139903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.139990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.140133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.140212 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.140383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.140462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.140635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.140728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.140864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.140944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.141097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.141176 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.141331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.141483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.141564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.141642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.141794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.141879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.142031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.142119 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.142268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.142346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.142500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.142577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.142732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.142821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.142963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.143080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.143763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.143844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.143998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.144076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.144231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.144307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.144469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.144576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.144722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.144802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.144954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.145036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.145185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.145271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.145416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.145494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.145646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.145728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.145908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.145986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.146143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.146220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.146373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.146457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.146605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.146683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.146841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.146919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.147075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.147155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.147307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.147385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.147537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.147616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.147764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.147844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.147991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.148069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.148218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.148296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.external = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.148451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.148579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.148701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.148781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.148937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.149018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.149171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.149250 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.password = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.149416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.149495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] auth.token = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.149649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.149727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.149899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.150010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.150149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.150229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.150384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.150463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.150611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.external = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.150695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.150838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.150919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.151100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.151184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.151340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.151419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.151575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.password = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.151662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.151802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] auth.token = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.151890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.152039 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.152120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.152278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.152359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.152541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.152658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.152771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.152855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.153018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.153101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.153260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.153340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.153487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.153579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.153720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.153799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.153963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.154042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.154207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.154294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.154452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.154533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.154683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.154764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.154923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.155010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.155157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.155238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.155414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.155493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.155661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.155810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.155885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.155956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.156129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.156209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.156378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.156465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.156654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.156736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.156910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.157000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.157148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.157235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.157399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.157477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.157632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.157711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.157866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.157953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.158105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.158183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.158347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.158425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.158579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.158666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.158847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.158925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.159080 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.159157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.159328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.159414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.159560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.159638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.159801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.159879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.160034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.160121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.160276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.160357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.160546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.160629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.160765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.160857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.160997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.161077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.161230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.161307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.161465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.161557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.161709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.161788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.161938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.162023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.162171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.162259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.162422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.162505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.162669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.162750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.162897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.162978 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.163123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.163202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.163349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.163435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.163579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.163659 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.163816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.163896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.164057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.164141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.164297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.164377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.164556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.164650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.164806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.164891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.165047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.165133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.165280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.165360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.165508 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.165588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.165741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.165823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.165968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.166047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.166200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.166280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.166436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.166517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.166667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.166749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.166907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.166987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.167133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.167213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.167377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.167457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.167613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.167692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.167843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.167923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.168085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.168171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.168329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.168408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.168587 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.168686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.168823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.168914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.169050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.169130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.169282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.169365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.169513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.169595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.169750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.169831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.169981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.170062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.170211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.170290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.170452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.170531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.170696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.170778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.170925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.171005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.171153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.171233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.171398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.171480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.171641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.171721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.171875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.171961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.172106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.172186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.172340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.172419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.172589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.172673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.172828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.172909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.173061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.173141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.173296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.173374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.173538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.173619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.173782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.173860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.174023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.174101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.174278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.174359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.174508 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.174587 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.174740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.174820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.174972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.175054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.175203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.175283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.175430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.175509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.175663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.175743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.175902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.175982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.176144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.176223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.176379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.176459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.176639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.176730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.176867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.176954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.177107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.177188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.177342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.177422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.177568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.177655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.177799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.177885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.178030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.178110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.178269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.178473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.178554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.178631 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.178739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.178820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.178969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.179058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.179194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.179274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.179435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.179514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.179666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.179754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.179900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.179979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.180132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.180211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.180374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.180462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.180634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.180717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.180865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.180945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.181098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.181191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.181335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] role.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.181414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.181578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.181656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] role.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.181808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.181895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.182043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.182122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.182293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.182372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.182538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.182624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.182773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.182852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.183015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.183095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.183248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.183335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.183483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.183562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.183715 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] role.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.183794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.183949 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.184036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] role.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.184176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.184255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.184419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.184499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.184690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.184782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.184933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.185012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.185176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.185322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.185402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.185561 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.185642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.185813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.185894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.186065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.186154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.186321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.186420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.186562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.186675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.186803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.186914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.187045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.187170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.187288 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.187425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.187665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.187752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.187899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.187979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.188134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.188213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.188368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.188456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.188647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.188730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.188906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.188986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.189156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.189248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.189397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.189478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.189636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.189717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.190397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.190492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.190627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.190706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.190860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.190939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.191090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.191179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.191333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.191411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.191581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.191660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.191827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.191915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.192063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.192142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.192296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.192374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.192550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.192650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.192796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.192877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.193033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.193111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.193272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.193361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.193503 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.193582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.193749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.193827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.193990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.194077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.194218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.194297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.194455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.194533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.194696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.194842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.194924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.195113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.195202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.195358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.195438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.195597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.195684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.195835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.195914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.196076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.196155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.196304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.196392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.196560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.196642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.196802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.196882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.197041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.197133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.197287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.197377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.197524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.197604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.197764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.197842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.197985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.198059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.198224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.198302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.198455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.198534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.198685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.198766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.198914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.198993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.199150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.199228 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.199381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.199462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.199535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80401) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 17:49:43.199606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.199839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.200081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.200313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.200570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.200802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.201037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.201272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 17:49:43.201437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 17:49:43.228877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.229110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.229337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.229483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.229713 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.229933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.243358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.243560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.268354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.268512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.268623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.268752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.320740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.320856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.370461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.370690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.371266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.371501 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.434525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.434748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.434835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.434966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.435053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.435182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.435265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.435394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.435486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.435603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.435694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.435880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.459381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.459605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.459820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.460030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.460237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.460394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.460478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.460654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.460740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.460879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.461099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.461317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.461532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.461749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.471819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.471916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.486194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.486345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.486429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.486504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.486636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.487245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.487419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.487553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.487637 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.487767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.487847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.487974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.488057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80401) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 17:49:43.505605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.505821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 17:49:43.505905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.506107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 17:49:43.506191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.506359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 17:49:43.506448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.506627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 17:49:43.506709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.506912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.507006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.507144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.507226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.507398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.507484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.507653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.518804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.519084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.519359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.519441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.519645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.519731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.519910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.520189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.526601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.527076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.529697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.529986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.530156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.530270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.530439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.530554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.530716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.530838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.530990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.531101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.531264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.531373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.531520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.531648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.531783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.531920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.532050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.532193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.532314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.532461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.532603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.532867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.555966 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.556231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.557492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.557743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.557889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.558021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.562408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.562610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.562692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.562891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.562974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.563172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.570224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.570424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.570510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.570704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.574677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.574799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.574928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.575086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.575214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.575365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.575497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.575647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.575795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.575934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.576095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.576219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.576368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.576490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.576672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.576767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.592814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.593007 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.593091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.593288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.597609 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.597870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.600771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.601064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.601146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.601447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.605426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.605871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.608610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.609085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.612342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.612667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.612830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.612961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.613132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.613253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.613421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.613529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.613717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.613799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.614009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.614293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.630440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.630732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.631003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.631272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.631504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.631798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.632063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.632323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.638214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.639200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.639803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.640082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.640799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.641075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.642912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.643234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.643535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.643828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.643911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.644112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.644196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.644430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.644738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.645017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.655578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.655866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.656154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.670858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.670957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.671056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.671137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.671214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.671289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.671364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.671439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.671523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.671523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.671523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.675760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.676365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.678769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.679341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.681776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.682288 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.683355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.683841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.684901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.685188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.685357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.685475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.685648 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.685933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.692002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.692356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.693662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.693977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.696640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.696916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.697050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.697207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.697292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.697488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.697622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.697763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.706062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.706191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.706422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.706506 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.706707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.706882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.713595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.713871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80401) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 17:49:43.713957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.714149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.714232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.714425 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.714517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.714696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.714780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.714969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.715053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.715238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.715331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.715512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.715596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.715786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.715868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.716143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.716241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.716421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.716523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.716714 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.716809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.716982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.717066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.717242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.717325 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.717505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80401) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 17:49:43.751580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55beb62509c0 pid: 80402 (default app) Aug 08 17:49:43.752177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55beb62509c0 pid: 80401 (default app) Aug 08 17:49:43.754940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59b35-23b1-441d-8125-a5221437c1b1 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:43.755222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59b35-23b1-441d-8125-a5221437c1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:43.755489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59b35-23b1-441d-8125-a5221437c1b1 None None] PATH_INFO: `/v3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:43.763218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1/1] 10.222.0.50 () {50 vars in 791 bytes} [Mon Aug 8 17:49:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:49:52.214874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b099415-8aaa-4945-98ac-ef1f1ac3051e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:52.215380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b099415-8aaa-4945-98ac-ef1f1ac3051e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:52.215380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b099415-8aaa-4945-98ac-ef1f1ac3051e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:52.219766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1/2] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:49:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:49:52.224809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:52.225241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:52.225350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:52.295045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: INFO dbcounter [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] Registered counter for database keystone Aug 08 17:49:52.314248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG oslo_db.sqlalchemy.engines [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80402) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 17:49:52.321027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writer thread running {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 17:49:52.497398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG passlib.handlers.bcrypt [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80402) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 17:49:52.497744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG passlib.handlers.bcrypt [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80402) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 17:49:52.552876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:49:52.554473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:49:52.685654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 2/3] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:49:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 17:49:52.853302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b099415-8aaa-4945-98ac-ef1f1ac3051e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:52.924515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: INFO dbcounter [None req-1b099415-8aaa-4945-98ac-ef1f1ac3051e None None] Registered counter for database keystone Aug 08 17:49:52.945331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG oslo_db.sqlalchemy.engines [None req-1b099415-8aaa-4945-98ac-ef1f1ac3051e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80401) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 17:49:52.951432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writer thread running {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 17:49:53.069187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b099415-8aaa-4945-98ac-ef1f1ac3051e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:53.073389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ff1c2-5585-49ad-9ba2-3f43e04cae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:53.073994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ff1c2-5585-49ad-9ba2-3f43e04cae8f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:53.074270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ff1c2-5585-49ad-9ba2-3f43e04cae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:53.074537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ff1c2-5585-49ad-9ba2-3f43e04cae8f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:53.083361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ff1c2-5585-49ad-9ba2-3f43e04cae8f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:53.134649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a10ff1c2-5585-49ad-9ba2-3f43e04cae8f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 17:49:53.134990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ff1c2-5585-49ad-9ba2-3f43e04cae8f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:53.144886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 2/4] 10.222.0.50 () {58 vars in 1324 bytes} [Mon Aug 8 17:49:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:49:53.150865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:53.165719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b1248-befd-4d2f-8645-5c10179ff4d6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:53.170177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f136d2-eacb-41b2-a849-4d7e5ae6dac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:53.171273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f136d2-eacb-41b2-a849-4d7e5ae6dac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:53.171552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f136d2-eacb-41b2-a849-4d7e5ae6dac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:53.171821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f136d2-eacb-41b2-a849-4d7e5ae6dac9 admin admin] PATH_INFO: `/v3/projects/6c1d22d0f80a4861a903f2c834f48928` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:53.173253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f136d2-eacb-41b2-a849-4d7e5ae6dac9 admin admin] RBAC: Authorizing `identity:get_project(project_id=6c1d22d0f80a4861a903f2c834f48928)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:53.224402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WARNING keystone.common.rbac_enforcer.enforcer [None req-77f136d2-eacb-41b2-a849-4d7e5ae6dac9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 17:49:53.224768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f136d2-eacb-41b2-a849-4d7e5ae6dac9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:53.229349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 3/5] 10.222.0.50 () {56 vars in 1207 bytes} [Mon Aug 8 17:49:53 2022] GET /identity/v3/projects/6c1d22d0f80a4861a903f2c834f48928 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:49:55.179780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0a7bf-7e6f-414e-a899-724f407e5ddb None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:55.180263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0a7bf-7e6f-414e-a899-724f407e5ddb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:55.180263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0a7bf-7e6f-414e-a899-724f407e5ddb None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:55.184297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 3/6] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:49:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:49:55.190117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7e2bd-0af1-4a0b-a45d-c020fc66afbb None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:55.190523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7e2bd-0af1-4a0b-a45d-c020fc66afbb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:55.190651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7e2bd-0af1-4a0b-a45d-c020fc66afbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:55.212261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-39d7e2bd-0af1-4a0b-a45d-c020fc66afbb None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:49:55.213810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-39d7e2bd-0af1-4a0b-a45d-c020fc66afbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:49:55.221879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 4/7] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:49:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 08 17:49:55.244077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0a7bf-7e6f-414e-a899-724f407e5ddb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:55.256593 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0a7bf-7e6f-414e-a899-724f407e5ddb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:55.260709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89de2d-4340-4eb7-ba29-c12be04b5c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:55.261293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89de2d-4340-4eb7-ba29-c12be04b5c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:55.261565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89de2d-4340-4eb7-ba29-c12be04b5c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:55.261827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89de2d-4340-4eb7-ba29-c12be04b5c05 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:55.269386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89de2d-4340-4eb7-ba29-c12be04b5c05 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:55.271149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89de2d-4340-4eb7-ba29-c12be04b5c05 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:55.280000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 4/8] 10.222.0.50 () {58 vars in 1324 bytes} [Mon Aug 8 17:49:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:49:55.285318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7e2bd-0af1-4a0b-a45d-c020fc66afbb None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:55.295006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7e2bd-0af1-4a0b-a45d-c020fc66afbb None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:55.300128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d25110-1bac-4403-a22c-483d4e5e9c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:55.300773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25110-1bac-4403-a22c-483d4e5e9c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:55.301054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25110-1bac-4403-a22c-483d4e5e9c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:55.301324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25110-1bac-4403-a22c-483d4e5e9c05 admin admin] PATH_INFO: `/v3/users/2c322d3415e942c2b44d2bbf25f886a1` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:55.302652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d25110-1bac-4403-a22c-483d4e5e9c05 admin admin] RBAC: Authorizing `identity:get_user(user_id=2c322d3415e942c2b44d2bbf25f886a1)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:55.304706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d25110-1bac-4403-a22c-483d4e5e9c05 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:55.306238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 5/9] 10.222.0.50 () {56 vars in 1198 bytes} [Mon Aug 8 17:49:55 2022] GET /identity/v3/users/2c322d3415e942c2b44d2bbf25f886a1 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:49:57.333664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a5f0b-5c81-4ab1-b79c-4ac3e6af834d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:57.334203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a5f0b-5c81-4ab1-b79c-4ac3e6af834d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:57.334203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a5f0b-5c81-4ab1-b79c-4ac3e6af834d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:57.335509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 5/10] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:49:57.340582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff449f80-afa9-40de-bd12-9e70334217a3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:57.340976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff449f80-afa9-40de-bd12-9e70334217a3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:57.341277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff449f80-afa9-40de-bd12-9e70334217a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:57.359380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ff449f80-afa9-40de-bd12-9e70334217a3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:49:57.360569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ff449f80-afa9-40de-bd12-9e70334217a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:49:57.398194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b62cb6-9883-4563-b0ac-40bc771c8e90 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:57.398551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b62cb6-9883-4563-b0ac-40bc771c8e90 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:57.398886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b62cb6-9883-4563-b0ac-40bc771c8e90 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:57.400261 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 6/11] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:49:57.405531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1eb7a-4a57-4efc-a8be-ac710c45b4a3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:57.405812 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1eb7a-4a57-4efc-a8be-ac710c45b4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:57.406086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1eb7a-4a57-4efc-a8be-ac710c45b4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:57.440137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 6/12] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:49:57.449441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG passlib.handlers.bcrypt [None req-aba1eb7a-4a57-4efc-a8be-ac710c45b4a3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80401) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 17:49:57.449864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG passlib.handlers.bcrypt [None req-aba1eb7a-4a57-4efc-a8be-ac710c45b4a3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80401) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 17:49:57.463117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff449f80-afa9-40de-bd12-9e70334217a3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:57.486440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:49:57.496082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-aba1eb7a-4a57-4efc-a8be-ac710c45b4a3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:49:57.497675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-aba1eb7a-4a57-4efc-a8be-ac710c45b4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:49:57.567185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff449f80-afa9-40de-bd12-9e70334217a3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:57.601946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 7/13] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:49:57.602278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351796c3-0728-4f15-830d-618add150434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:57.602944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351796c3-0728-4f15-830d-618add150434 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:57.603194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351796c3-0728-4f15-830d-618add150434 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:57.603464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351796c3-0728-4f15-830d-618add150434 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:57.615050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351796c3-0728-4f15-830d-618add150434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:49:57.623286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1eb7a-4a57-4efc-a8be-ac710c45b4a3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-351796c3-0728-4f15-830d-618add150434 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:49:57.623922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:49:57.625003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:49:57.626009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 7/14] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:49:57.631377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351796c3-0728-4f15-830d-618add150434 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:57.712458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:49:57.724611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1eb7a-4a57-4efc-a8be-ac710c45b4a3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:57.726883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:49:57.730267 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351796c3-0728-4f15-830d-618add150434 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:57.762988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24d201-5546-4a05-9c9b-3eb4d3ef29eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:57.763826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24d201-5546-4a05-9c9b-3eb4d3ef29eb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:57.764103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24d201-5546-4a05-9c9b-3eb4d3ef29eb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:57.764369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24d201-5546-4a05-9c9b-3eb4d3ef29eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:57.764872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db514d2-656c-4e40-abf9-2310ca557f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:57.765464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db514d2-656c-4e40-abf9-2310ca557f6b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:57.765759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db514d2-656c-4e40-abf9-2310ca557f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:57.766028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db514d2-656c-4e40-abf9-2310ca557f6b None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:57.766687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db514d2-656c-4e40-abf9-2310ca557f6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:49:57.766990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db514d2-656c-4e40-abf9-2310ca557f6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:57.768374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db514d2-656c-4e40-abf9-2310ca557f6b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:57.779912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 8/15] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:49:57.785894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db514d2-656c-4e40-abf9-2310ca557f6b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:57.832736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24d201-5546-4a05-9c9b-3eb4d3ef29eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:57.834284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24d201-5546-4a05-9c9b-3eb4d3ef29eb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:57.888136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db514d2-656c-4e40-abf9-2310ca557f6b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:57.942896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 8/16] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 17:49:57.949064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24d201-5546-4a05-9c9b-3eb4d3ef29eb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:57.952396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:49:58.036857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0124f4-0e62-4dd6-acd2-7cce4b84bd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:58.037667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0124f4-0e62-4dd6-acd2-7cce4b84bd81 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:58.038049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0124f4-0e62-4dd6-acd2-7cce4b84bd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:58.038388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0124f4-0e62-4dd6-acd2-7cce4b84bd81 None admin] PATH_INFO: `/v3/users/2c322d3415e942c2b44d2bbf25f886a1` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:58.039891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0124f4-0e62-4dd6-acd2-7cce4b84bd81 None admin] RBAC: Authorizing `identity:get_user(user_id=2c322d3415e942c2b44d2bbf25f886a1)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:58.041825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0124f4-0e62-4dd6-acd2-7cce4b84bd81 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:58.044161 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 9/17] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:49:57 2022] GET /identity/v3/users/2c322d3415e942c2b44d2bbf25f886a1 => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:49:58.049879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0124f4-0e62-4dd6-acd2-7cce4b84bd81 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:58.065230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:49:58.081760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24d201-5546-4a05-9c9b-3eb4d3ef29eb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:58.117119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca30ebd-dd89-4ef3-a14f-176066ed49cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:58.118077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30ebd-dd89-4ef3-a14f-176066ed49cf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:58.118535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30ebd-dd89-4ef3-a14f-176066ed49cf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:58.118949 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30ebd-dd89-4ef3-a14f-176066ed49cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:58.137167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca30ebd-dd89-4ef3-a14f-176066ed49cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:58.138706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca30ebd-dd89-4ef3-a14f-176066ed49cf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:58.151964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0124f4-0e62-4dd6-acd2-7cce4b84bd81 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6ca30ebd-dd89-4ef3-a14f-176066ed49cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:49:58.156620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:49:58.157675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:49:58.159035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:49:58.159035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:49:58.159035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:49:58.159035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 9/18] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:49:58.164043 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca30ebd-dd89-4ef3-a14f-176066ed49cf None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:58.192524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a515ba-6522-4881-855d-24ceec63cee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:58.193161 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a515ba-6522-4881-855d-24ceec63cee2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:58.193431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a515ba-6522-4881-855d-24ceec63cee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:58.193694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a515ba-6522-4881-855d-24ceec63cee2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:58.194800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a515ba-6522-4881-855d-24ceec63cee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:58.196236 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a515ba-6522-4881-855d-24ceec63cee2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:58.197802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 10/19] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:49:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:49:58.203563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a515ba-6522-4881-855d-24ceec63cee2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:58.224053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:49:58.266646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca30ebd-dd89-4ef3-a14f-176066ed49cf None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:58.283513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:49:58.301525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b955f89-4f0a-4da6-bbbd-eb173099b6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:58.301813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a515ba-6522-4881-855d-24ceec63cee2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:58.302149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b955f89-4f0a-4da6-bbbd-eb173099b6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:58.302428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b955f89-4f0a-4da6-bbbd-eb173099b6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:58.302695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b955f89-4f0a-4da6-bbbd-eb173099b6b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:58.303332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b955f89-4f0a-4da6-bbbd-eb173099b6b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:49:58.303631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b955f89-4f0a-4da6-bbbd-eb173099b6b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:58.305081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b955f89-4f0a-4da6-bbbd-eb173099b6b2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:58.324471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 10/20] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:49:58.336933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b4278-a572-4e07-8a8f-06fadd71a632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:58.337560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b4278-a572-4e07-8a8f-06fadd71a632 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:58.337834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b4278-a572-4e07-8a8f-06fadd71a632 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:58.338101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b4278-a572-4e07-8a8f-06fadd71a632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:58.338846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b4278-a572-4e07-8a8f-06fadd71a632 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.domain.id=default, user.id=2c322d3415e942c2b44d2bbf25f886a1) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:49:58.339147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b4278-a572-4e07-8a8f-06fadd71a632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:58.340693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b4278-a572-4e07-8a8f-06fadd71a632 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:58.349172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 11/21] 10.222.0.50 () {56 vars in 1322 bytes} [Mon Aug 8 17:49:58 2022] GET /identity/v3/role_assignments?user.id=2c322d3415e942c2b44d2bbf25f886a1&scope.domain.id=default&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:00.661082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c075de1-1a7e-4695-bfee-a25ae27fe60c None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:00.661591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c075de1-1a7e-4695-bfee-a25ae27fe60c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:00.661696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c075de1-1a7e-4695-bfee-a25ae27fe60c None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:00.663218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 11/22] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:00.668045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da72b53-9b8f-4701-b26b-2471737942a4 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:00.668958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da72b53-9b8f-4701-b26b-2471737942a4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:00.669426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da72b53-9b8f-4701-b26b-2471737942a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:00.693495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7da72b53-9b8f-4701-b26b-2471737942a4 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:00.695052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7da72b53-9b8f-4701-b26b-2471737942a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:00.763373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a3b3c-f459-4d87-b5e3-9f372e1aaed0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:00.763984 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a3b3c-f459-4d87-b5e3-9f372e1aaed0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:00.764447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a3b3c-f459-4d87-b5e3-9f372e1aaed0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:00.766001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 12/23] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:00.778944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 12/24] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:00.780286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e990c6-74b4-4cda-af72-334c84fcfeba None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:00.780801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e990c6-74b4-4cda-af72-334c84fcfeba None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:00.781278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e990c6-74b4-4cda-af72-334c84fcfeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:00.812339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da72b53-9b8f-4701-b26b-2471737942a4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:00.822539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c4e990c6-74b4-4cda-af72-334c84fcfeba None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:00.824006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c4e990c6-74b4-4cda-af72-334c84fcfeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:00.928255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:00.958228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da72b53-9b8f-4701-b26b-2471737942a4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:00.996560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 13/25] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:00.996768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6910a21-8ff1-4220-af01-a4483eb71e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:00.997260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6910a21-8ff1-4220-af01-a4483eb71e7e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:00.997533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6910a21-8ff1-4220-af01-a4483eb71e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:00.997924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6910a21-8ff1-4220-af01-a4483eb71e7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.009216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6910a21-8ff1-4220-af01-a4483eb71e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f6910a21-8ff1-4220-af01-a4483eb71e7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:01.014038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:01.014891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:01.015675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 13/26] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:01.022294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e990c6-74b4-4cda-af72-334c84fcfeba None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:01.034412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6910a21-8ff1-4220-af01-a4483eb71e7e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:01.124874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:01.151389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6910a21-8ff1-4220-af01-a4483eb71e7e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:01.193338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b3783-e1bb-476c-8157-8f875de8027a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:01.193866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b3783-e1bb-476c-8157-8f875de8027a None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:01.194140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b3783-e1bb-476c-8157-8f875de8027a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:01.194404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b3783-e1bb-476c-8157-8f875de8027a None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.195078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b3783-e1bb-476c-8157-8f875de8027a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:01.196464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b3783-e1bb-476c-8157-8f875de8027a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:01.200713 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:01.240272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e990c6-74b4-4cda-af72-334c84fcfeba None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:01.277552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279ff74-8ae6-4985-afd3-9aeccd1c22a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:01.278168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279ff74-8ae6-4985-afd3-9aeccd1c22a3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:01.278441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279ff74-8ae6-4985-afd3-9aeccd1c22a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:01.278708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279ff74-8ae6-4985-afd3-9aeccd1c22a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.279365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279ff74-8ae6-4985-afd3-9aeccd1c22a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:01.279660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279ff74-8ae6-4985-afd3-9aeccd1c22a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:01.281178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279ff74-8ae6-4985-afd3-9aeccd1c22a3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:01.293528 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 14/27] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:01.298123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e789cf-fabd-4617-9198-2140aeacfe63 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:01.298123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e789cf-fabd-4617-9198-2140aeacfe63 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:01.298123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e789cf-fabd-4617-9198-2140aeacfe63 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.298123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 15/28] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:01.298123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c95337-dc6d-45f0-af45-ef5ef0881481 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:01.298123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c95337-dc6d-45f0-af45-ef5ef0881481 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:01.299048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c95337-dc6d-45f0-af45-ef5ef0881481 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.299923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 16/29] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:01.304039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c95337-dc6d-45f0-af45-ef5ef0881481 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:01.421417 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 14/30] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 17:50:01 2022] POST /identity/v3/roles => generated 219 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:50:01.423107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee186381-fa06-47ef-a253-5671034d8132 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:01.423929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee186381-fa06-47ef-a253-5671034d8132 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:01.424203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee186381-fa06-47ef-a253-5671034d8132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.445259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ee186381-fa06-47ef-a253-5671034d8132 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:01.446810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ee186381-fa06-47ef-a253-5671034d8132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:01.535571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 15/31] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:01.538118 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703db3fc-bfa2-4cc6-9a6f-4261bea9d9a0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:01.538638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703db3fc-bfa2-4cc6-9a6f-4261bea9d9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:01.539064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703db3fc-bfa2-4cc6-9a6f-4261bea9d9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.553018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c95337-dc6d-45f0-af45-ef5ef0881481 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:01.565587 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:01.569887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-703db3fc-bfa2-4cc6-9a6f-4261bea9d9a0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:01.572085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-703db3fc-bfa2-4cc6-9a6f-4261bea9d9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:01.601691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e2417-0a85-4b92-8287-6318e95d88ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:01.602485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e2417-0a85-4b92-8287-6318e95d88ed None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:01.602854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e2417-0a85-4b92-8287-6318e95d88ed None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:01.603218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e2417-0a85-4b92-8287-6318e95d88ed None admin] PATH_INFO: `/v3/users/2c322d3415e942c2b44d2bbf25f886a1` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.606565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e2417-0a85-4b92-8287-6318e95d88ed None admin] RBAC: Authorizing `identity:get_user(user_id=2c322d3415e942c2b44d2bbf25f886a1)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:01.608763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e2417-0a85-4b92-8287-6318e95d88ed None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:01.612630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 17/32] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:01 2022] GET /identity/v3/users/2c322d3415e942c2b44d2bbf25f886a1 => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:01.613436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e2417-0a85-4b92-8287-6318e95d88ed None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:01.685245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 16/33] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:01.686702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703db3fc-bfa2-4cc6-9a6f-4261bea9d9a0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:01.758523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e2417-0a85-4b92-8287-6318e95d88ed None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:01.772327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:01.800694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b73df-bc6e-4d49-a343-c401a71e84fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:01.801449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b73df-bc6e-4d49-a343-c401a71e84fb None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:01.801824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b73df-bc6e-4d49-a343-c401a71e84fb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:01.802185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b73df-bc6e-4d49-a343-c401a71e84fb None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.803024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b73df-bc6e-4d49-a343-c401a71e84fb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:01.804599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b73df-bc6e-4d49-a343-c401a71e84fb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:01.829206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703db3fc-bfa2-4cc6-9a6f-4261bea9d9a0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:01.871753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bdae0-8993-4e26-bd69-8b5c5718a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:01.872386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bdae0-8993-4e26-bd69-8b5c5718a1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:01.872693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bdae0-8993-4e26-bd69-8b5c5718a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:01.872962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bdae0-8993-4e26-bd69-8b5c5718a1c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:01.874212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bdae0-8993-4e26-bd69-8b5c5718a1c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:01.875774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bdae0-8993-4e26-bd69-8b5c5718a1c0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:01.877633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 17/34] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:01.878190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bdae0-8993-4e26-bd69-8b5c5718a1c0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:02.001424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bdae0-8993-4e26-bd69-8b5c5718a1c0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:02.026865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:02.034920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 18/35] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 17:50:01 2022] POST /identity/v3/roles => generated 225 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:02.035949 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b73df-bc6e-4d49-a343-c401a71e84fb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:02.040209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24f401-48cb-4581-8214-cb863d2bbc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:02.042026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24f401-48cb-4581-8214-cb863d2bbc5e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.042459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24f401-48cb-4581-8214-cb863d2bbc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.042899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24f401-48cb-4581-8214-cb863d2bbc5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.057597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24f401-48cb-4581-8214-cb863d2bbc5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:02.062746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24f401-48cb-4581-8214-cb863d2bbc5e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5b24f401-48cb-4581-8214-cb863d2bbc5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:50:02.093770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:02.094696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:50:02.096175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:50:02.096175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:50:02.096175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:02.096175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 18/36] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:02.105332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:02.106140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24f401-48cb-4581-8214-cb863d2bbc5e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:02.207846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b73df-bc6e-4d49-a343-c401a71e84fb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:02.257667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e8f28-2ad8-4add-995d-be3fc7f5f5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:02.258338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e8f28-2ad8-4add-995d-be3fc7f5f5a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.258436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24f401-48cb-4581-8214-cb863d2bbc5e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:02.258621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e8f28-2ad8-4add-995d-be3fc7f5f5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.258898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e8f28-2ad8-4add-995d-be3fc7f5f5a0 None admin] PATH_INFO: `/v3/domains/default/users/2c322d3415e942c2b44d2bbf25f886a1/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.261480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e8f28-2ad8-4add-995d-be3fc7f5f5a0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2c322d3415e942c2b44d2bbf25f886a1, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:02.262600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e8f28-2ad8-4add-995d-be3fc7f5f5a0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:02.301910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892d077-9830-406e-bdab-e1c417e4235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:02.302586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892d077-9830-406e-bdab-e1c417e4235e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.302865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892d077-9830-406e-bdab-e1c417e4235e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.303137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892d077-9830-406e-bdab-e1c417e4235e None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.303788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892d077-9830-406e-bdab-e1c417e4235e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:02.304088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892d077-9830-406e-bdab-e1c417e4235e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:02.305600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892d077-9830-406e-bdab-e1c417e4235e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:02.323684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 19/37] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:02.325018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c297f3-6027-4b3f-bf88-c2a28de67bf5 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.325311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c297f3-6027-4b3f-bf88-c2a28de67bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.325583 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c297f3-6027-4b3f-bf88-c2a28de67bf5 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.326923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 20/38] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:02.329891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c297f3-6027-4b3f-bf88-c2a28de67bf5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:02.366471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:02.366642 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 19/39] 10.222.0.50 () {58 vars in 1361 bytes} [Mon Aug 8 17:50:02 2022] PUT /identity/v3/domains/default/users/2c322d3415e942c2b44d2bbf25f886a1/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:02.367911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf3a45-bad0-4c26-a16b-70ab5b71b4ee None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.368195 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf3a45-bad0-4c26-a16b-70ab5b71b4ee None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.368468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf3a45-bad0-4c26-a16b-70ab5b71b4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.388625 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4dbf3a45-bad0-4c26-a16b-70ab5b71b4ee None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:02.390103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4dbf3a45-bad0-4c26-a16b-70ab5b71b4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:02.455820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c297f3-6027-4b3f-bf88-c2a28de67bf5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:02.470009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 20/40] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:02.471676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c137e52-da87-4b94-ae17-9a9f0642d25c None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.472145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c137e52-da87-4b94-ae17-9a9f0642d25c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.472591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c137e52-da87-4b94-ae17-9a9f0642d25c None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.474049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 21/41] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:02.479863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b569c7a-cfcc-4876-8ed1-6d0e5ed84e0f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.480420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b569c7a-cfcc-4876-8ed1-6d0e5ed84e0f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.480921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b569c7a-cfcc-4876-8ed1-6d0e5ed84e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.502142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1b569c7a-cfcc-4876-8ed1-6d0e5ed84e0f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:02.502448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5dbf3-d3a1-4c68-b615-aebcec415e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:02.503194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5dbf3-d3a1-4c68-b615-aebcec415e85 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.503373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1b569c7a-cfcc-4876-8ed1-6d0e5ed84e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:02.503591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5dbf3-d3a1-4c68-b615-aebcec415e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.503936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5dbf3-d3a1-4c68-b615-aebcec415e85 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.504696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5dbf3-d3a1-4c68-b615-aebcec415e85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:02.506261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5dbf3-d3a1-4c68-b615-aebcec415e85 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:02.531114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:02.576209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 22/42] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:02.576988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b569c7a-cfcc-4876-8ed1-6d0e5ed84e0f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:02.620363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 21/43] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:50:02 2022] POST /identity/v3/projects => generated 301 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:50:02.621618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5dbf3-d3a1-4c68-b615-aebcec415e85 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:02.701921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b569c7a-cfcc-4876-8ed1-6d0e5ed84e0f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:02.745139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:02.753419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95d5a9-2d39-4a9d-b7b3-f5fe172c4d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:02.754736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95d5a9-2d39-4a9d-b7b3-f5fe172c4d22 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.755380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95d5a9-2d39-4a9d-b7b3-f5fe172c4d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.760091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95d5a9-2d39-4a9d-b7b3-f5fe172c4d22 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.760091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95d5a9-2d39-4a9d-b7b3-f5fe172c4d22 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:02.760091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95d5a9-2d39-4a9d-b7b3-f5fe172c4d22 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:02.763289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5dbf3-d3a1-4c68-b615-aebcec415e85 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:02.807965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db8e21-1896-4cd9-8d8b-d8459d26285e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:02.807965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db8e21-1896-4cd9-8d8b-d8459d26285e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.808199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db8e21-1896-4cd9-8d8b-d8459d26285e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.808459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db8e21-1896-4cd9-8d8b-d8459d26285e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.809697 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db8e21-1896-4cd9-8d8b-d8459d26285e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:02.811219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db8e21-1896-4cd9-8d8b-d8459d26285e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:02.814539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 22/44] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:02.819419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db8e21-1896-4cd9-8d8b-d8459d26285e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:02.840935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 23/45] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 17:50:02 2022] POST /identity/v3/roles => generated 223 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:02.935415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db8e21-1896-4cd9-8d8b-d8459d26285e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:02.971853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63c6e6-abaf-4324-8911-ab306e4861f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:02.972471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63c6e6-abaf-4324-8911-ab306e4861f2 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:02.972780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63c6e6-abaf-4324-8911-ab306e4861f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:02.973054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63c6e6-abaf-4324-8911-ab306e4861f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:02.973710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63c6e6-abaf-4324-8911-ab306e4861f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:02.975178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63c6e6-abaf-4324-8911-ab306e4861f2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:03.058171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 23/46] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:50:02 2022] POST /identity/v3/projects => generated 312 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:50:04.372889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01fc6-eea5-4797-ab15-266d0d04bdcb None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:04.373386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01fc6-eea5-4797-ab15-266d0d04bdcb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:04.373468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01fc6-eea5-4797-ab15-266d0d04bdcb None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:04.374787 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 24/47] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:04.380496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77224392-5840-4f6a-a70d-a9af2fbc60ae None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:04.380912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77224392-5840-4f6a-a70d-a9af2fbc60ae None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:04.381067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77224392-5840-4f6a-a70d-a9af2fbc60ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:04.393864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:04.398955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-77224392-5840-4f6a-a70d-a9af2fbc60ae None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:04.400142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-77224392-5840-4f6a-a70d-a9af2fbc60ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:04.464415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 24/48] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:04.487073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01fc6-eea5-4797-ab15-266d0d04bdcb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:04.506194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:04.587641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01fc6-eea5-4797-ab15-266d0d04bdcb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:04.621979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb571a7c-9dc6-4b94-9814-de0035781d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:04.622606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb571a7c-9dc6-4b94-9814-de0035781d18 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:04.622882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb571a7c-9dc6-4b94-9814-de0035781d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:04.623200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb571a7c-9dc6-4b94-9814-de0035781d18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:04.634202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb571a7c-9dc6-4b94-9814-de0035781d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-eb571a7c-9dc6-4b94-9814-de0035781d18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:04.640529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:04.641536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:04.642307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 25/49] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:04.647474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77224392-5840-4f6a-a70d-a9af2fbc60ae None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:04.744648 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77224392-5840-4f6a-a70d-a9af2fbc60ae None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:04.777277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2b4a2-2828-425d-90ee-3df0fd51aa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:04.777877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2b4a2-2828-425d-90ee-3df0fd51aa57 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:04.778153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2b4a2-2828-425d-90ee-3df0fd51aa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:04.778424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2b4a2-2828-425d-90ee-3df0fd51aa57 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:04.779089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2b4a2-2828-425d-90ee-3df0fd51aa57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:04.779387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2b4a2-2828-425d-90ee-3df0fd51aa57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:04.780819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2b4a2-2828-425d-90ee-3df0fd51aa57 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:04.783811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:04.790272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 25/50] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:04.796194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb571a7c-9dc6-4b94-9814-de0035781d18 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:04.875205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:04.896290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb571a7c-9dc6-4b94-9814-de0035781d18 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:04.929945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c020cc4-c839-48a3-b490-219c497e09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:04.930564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c020cc4-c839-48a3-b490-219c497e09da None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:04.930844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c020cc4-c839-48a3-b490-219c497e09da None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:04.931118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c020cc4-c839-48a3-b490-219c497e09da None admin] PATH_INFO: `/v3/users/2c322d3415e942c2b44d2bbf25f886a1` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:04.932246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c020cc4-c839-48a3-b490-219c497e09da None admin] RBAC: Authorizing `identity:get_user(user_id=2c322d3415e942c2b44d2bbf25f886a1)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:04.933739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c020cc4-c839-48a3-b490-219c497e09da None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:04.935318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 26/51] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:04 2022] GET /identity/v3/users/2c322d3415e942c2b44d2bbf25f886a1 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:04.940910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2b4a2-2828-425d-90ee-3df0fd51aa57 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:05.039376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2b4a2-2828-425d-90ee-3df0fd51aa57 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:05.047095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24912786-33d9-4f8b-8e5e-bccbbb19329e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:05.047380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24912786-33d9-4f8b-8e5e-bccbbb19329e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:05.047657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24912786-33d9-4f8b-8e5e-bccbbb19329e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:05.049064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 27/52] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:05.053738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962aedb-3936-4364-a2fe-7efcea875384 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:05.054021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962aedb-3936-4364-a2fe-7efcea875384 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:05.054292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962aedb-3936-4364-a2fe-7efcea875384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:05.071623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9962aedb-3936-4364-a2fe-7efcea875384 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:05.072847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9962aedb-3936-4364-a2fe-7efcea875384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:05.073644 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13cc28-4e46-4686-a52c-60ef96312c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:05.074269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13cc28-4e46-4686-a52c-60ef96312c64 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:05.074879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13cc28-4e46-4686-a52c-60ef96312c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:05.074879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13cc28-4e46-4686-a52c-60ef96312c64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:05.075921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13cc28-4e46-4686-a52c-60ef96312c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:05.077397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13cc28-4e46-4686-a52c-60ef96312c64 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:05.078961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 26/53] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:05.085505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13cc28-4e46-4686-a52c-60ef96312c64 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:05.139512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 28/54] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:05.148332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:05.162582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962aedb-3936-4364-a2fe-7efcea875384 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:05.182173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13cc28-4e46-4686-a52c-60ef96312c64 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:05.221052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8e71e-db28-4602-8c95-f38adc7fad1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:05.221404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8e71e-db28-4602-8c95-f38adc7fad1f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:05.221688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8e71e-db28-4602-8c95-f38adc7fad1f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:05.221953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8e71e-db28-4602-8c95-f38adc7fad1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:05.222666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8e71e-db28-4602-8c95-f38adc7fad1f None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.domain.id=default, user.id=2c322d3415e942c2b44d2bbf25f886a1) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:05.222976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8e71e-db28-4602-8c95-f38adc7fad1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:05.224421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8e71e-db28-4602-8c95-f38adc7fad1f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:05.229991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 27/55] 10.222.0.50 () {56 vars in 1322 bytes} [Mon Aug 8 17:50:05 2022] GET /identity/v3/role_assignments?user.id=2c322d3415e942c2b44d2bbf25f886a1&scope.domain.id=default&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:05.236792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:05.262864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962aedb-3936-4364-a2fe-7efcea875384 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:05.300396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d98a5-c996-4d27-acff-1ac99ac1500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:05.301064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d98a5-c996-4d27-acff-1ac99ac1500b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:05.301337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d98a5-c996-4d27-acff-1ac99ac1500b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:05.301600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d98a5-c996-4d27-acff-1ac99ac1500b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:05.302738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d98a5-c996-4d27-acff-1ac99ac1500b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:05.304198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d98a5-c996-4d27-acff-1ac99ac1500b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:05.306063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 29/56] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:05.311851 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8e71e-db28-4602-8c95-f38adc7fad1f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:05.415416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8e71e-db28-4602-8c95-f38adc7fad1f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:05.443007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:05.448424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3be2b9-e699-4ccf-ac29-103d2d4f8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:05.449049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3be2b9-e699-4ccf-ac29-103d2d4f8c16 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:05.449326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3be2b9-e699-4ccf-ac29-103d2d4f8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:05.449595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3be2b9-e699-4ccf-ac29-103d2d4f8c16 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:05.450227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3be2b9-e699-4ccf-ac29-103d2d4f8c16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:05.451679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3be2b9-e699-4ccf-ac29-103d2d4f8c16 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:05.532142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 28/57] 10.222.0.50 () {60 vars in 1140 bytes} [Mon Aug 8 17:50:05 2022] POST /identity/v3/projects => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:07.517539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5b906-9c26-4058-aece-11d64027573e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:07.518051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5b906-9c26-4058-aece-11d64027573e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:07.518155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5b906-9c26-4058-aece-11d64027573e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:07.519417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 30/58] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:07.524665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c176d27-421b-45dc-b57b-9213e2167799 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:07.525072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c176d27-421b-45dc-b57b-9213e2167799 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:07.525208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c176d27-421b-45dc-b57b-9213e2167799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:07.542469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3c176d27-421b-45dc-b57b-9213e2167799 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:07.543636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3c176d27-421b-45dc-b57b-9213e2167799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:07.607319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 29/59] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:07.631669 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5b906-9c26-4058-aece-11d64027573e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:07.732055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5b906-9c26-4058-aece-11d64027573e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:07.765240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15a5eb-fe55-4303-bdb4-bbee5d113398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:07.765854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15a5eb-fe55-4303-bdb4-bbee5d113398 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:07.766130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15a5eb-fe55-4303-bdb4-bbee5d113398 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:07.766400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15a5eb-fe55-4303-bdb4-bbee5d113398 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:07.767594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15a5eb-fe55-4303-bdb4-bbee5d113398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:07.769026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15a5eb-fe55-4303-bdb4-bbee5d113398 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:07.770572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 31/60] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:07.776310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c176d27-421b-45dc-b57b-9213e2167799 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:07.872187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c176d27-421b-45dc-b57b-9213e2167799 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:07.884658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:07.904252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db2771-1e16-4fa7-acd1-7a280cad1890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:07.904980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db2771-1e16-4fa7-acd1-7a280cad1890 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:07.905260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db2771-1e16-4fa7-acd1-7a280cad1890 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:07.905576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db2771-1e16-4fa7-acd1-7a280cad1890 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:07.906217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db2771-1e16-4fa7-acd1-7a280cad1890 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:07.908410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db2771-1e16-4fa7-acd1-7a280cad1890 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:08.023971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 30/61] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:50:07 2022] POST /identity/v3/users => generated 275 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:11.082154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc030-b9f6-4a54-b141-9aaf1ca52852 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.083152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc030-b9f6-4a54-b141-9aaf1ca52852 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.083650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc030-b9f6-4a54-b141-9aaf1ca52852 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.085219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 32/62] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:11.090604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ab66-c399-4ec6-ad4b-8ea39cb6d776 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.091014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ab66-c399-4ec6-ad4b-8ea39cb6d776 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.091156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ab66-c399-4ec6-ad4b-8ea39cb6d776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.113041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1b40ab66-c399-4ec6-ad4b-8ea39cb6d776 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:11.114477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1b40ab66-c399-4ec6-ad4b-8ea39cb6d776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:11.130819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c527db8-f177-4ea2-abfb-aab5ce65686a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.131405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c527db8-f177-4ea2-abfb-aab5ce65686a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.132003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c527db8-f177-4ea2-abfb-aab5ce65686a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.133856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 33/63] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:11.138469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025daccb-6139-439f-bfd4-375b7089a467 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.139001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025daccb-6139-439f-bfd4-375b7089a467 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.139522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025daccb-6139-439f-bfd4-375b7089a467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.166994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-025daccb-6139-439f-bfd4-375b7089a467 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:11.168641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-025daccb-6139-439f-bfd4-375b7089a467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:11.178345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:11.193554 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 31/64] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:11.217192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40ab66-c399-4ec6-ad4b-8ea39cb6d776 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:11.289839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 34/65] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:11.334416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025daccb-6139-439f-bfd4-375b7089a467 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:11.345627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:11.352474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40ab66-c399-4ec6-ad4b-8ea39cb6d776 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:11.390599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240108fd-2e25-48a8-bfea-7999a62e93a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:11.391634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240108fd-2e25-48a8-bfea-7999a62e93a1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.392106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240108fd-2e25-48a8-bfea-7999a62e93a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.392602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240108fd-2e25-48a8-bfea-7999a62e93a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.403598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240108fd-2e25-48a8-bfea-7999a62e93a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-240108fd-2e25-48a8-bfea-7999a62e93a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:11.413432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:11.414331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:11.415145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 32/66] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:11.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240108fd-2e25-48a8-bfea-7999a62e93a1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:11.526753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025daccb-6139-439f-bfd4-375b7089a467 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:11.527602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240108fd-2e25-48a8-bfea-7999a62e93a1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:11.563771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed49575-916e-4471-a21e-3b3004170a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:11.564391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed49575-916e-4471-a21e-3b3004170a6e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.564877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed49575-916e-4471-a21e-3b3004170a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.565296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed49575-916e-4471-a21e-3b3004170a6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.567304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1652af9-de73-4694-b1e9-03d1081f6ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:11.568249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1652af9-de73-4694-b1e9-03d1081f6ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.568768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1652af9-de73-4694-b1e9-03d1081f6ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.569251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1652af9-de73-4694-b1e9-03d1081f6ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.570112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1652af9-de73-4694-b1e9-03d1081f6ee3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:11.570611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1652af9-de73-4694-b1e9-03d1081f6ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:11.572193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1652af9-de73-4694-b1e9-03d1081f6ee3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:11.573503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:11.576827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed49575-916e-4471-a21e-3b3004170a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-fed49575-916e-4471-a21e-3b3004170a6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:11.582476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:11.583359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:11.584137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:11.585225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 33/67] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:11.586495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 35/68] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:11.589152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0380f69-dd22-4f25-9631-ee2bde29a0b3 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.589483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed49575-916e-4471-a21e-3b3004170a6e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:11.589589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0380f69-dd22-4f25-9631-ee2bde29a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.589837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0380f69-dd22-4f25-9631-ee2bde29a0b3 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.591223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 34/69] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:11.592319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0380f69-dd22-4f25-9631-ee2bde29a0b3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:11.616443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:11.701605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed49575-916e-4471-a21e-3b3004170a6e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:11.703329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0380f69-dd22-4f25-9631-ee2bde29a0b3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:11.738407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5994309-1a64-4770-8831-5991467a382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:11.739035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5994309-1a64-4770-8831-5991467a382c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.739553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5994309-1a64-4770-8831-5991467a382c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.739553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5994309-1a64-4770-8831-5991467a382c None admin] PATH_INFO: `/v3/users/2c322d3415e942c2b44d2bbf25f886a1` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.740804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5994309-1a64-4770-8831-5991467a382c None admin] RBAC: Authorizing `identity:get_user(user_id=2c322d3415e942c2b44d2bbf25f886a1)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:11.742265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5994309-1a64-4770-8831-5991467a382c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:11.743835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 36/70] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:11 2022] GET /identity/v3/users/2c322d3415e942c2b44d2bbf25f886a1 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:11.748274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285dd5-123b-4f1b-90f7-09111667ce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:11.748274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285dd5-123b-4f1b-90f7-09111667ce96 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.748274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285dd5-123b-4f1b-90f7-09111667ce96 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.748274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285dd5-123b-4f1b-90f7-09111667ce96 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.748615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285dd5-123b-4f1b-90f7-09111667ce96 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:11.748615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285dd5-123b-4f1b-90f7-09111667ce96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:11.748615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285dd5-123b-4f1b-90f7-09111667ce96 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:11.749985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d73f45-2bf1-4052-b7bd-f8b736e79f5a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.750273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d73f45-2bf1-4052-b7bd-f8b736e79f5a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.751024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d73f45-2bf1-4052-b7bd-f8b736e79f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.757458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 35/71] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:11.761957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c721ea-0777-4084-8e81-d9517809eca6 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.761957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c721ea-0777-4084-8e81-d9517809eca6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.761957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c721ea-0777-4084-8e81-d9517809eca6 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.761957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 36/72] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:11.764842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c721ea-0777-4084-8e81-d9517809eca6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:11.771731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f9d73f45-2bf1-4052-b7bd-f8b736e79f5a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:11.772979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f9d73f45-2bf1-4052-b7bd-f8b736e79f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:11.843563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:11.850885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 37/73] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:11.854660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d73f45-2bf1-4052-b7bd-f8b736e79f5a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:11.859355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:11.878570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c721ea-0777-4084-8e81-d9517809eca6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:11.913591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c8ae4-5a87-4efa-b535-0f4006bef558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:11.914172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c8ae4-5a87-4efa-b535-0f4006bef558 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.914452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c8ae4-5a87-4efa-b535-0f4006bef558 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.914719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c8ae4-5a87-4efa-b535-0f4006bef558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:11.962154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d73f45-2bf1-4052-b7bd-f8b736e79f5a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:11.980835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c8ae4-5a87-4efa-b535-0f4006bef558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:11.982292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c8ae4-5a87-4efa-b535-0f4006bef558 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:11.995435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffec4a-8ad6-4384-bb55-adcb6a7fd7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:11.996101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffec4a-8ad6-4384-bb55-adcb6a7fd7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:11.996380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffec4a-8ad6-4384-bb55-adcb6a7fd7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:11.996684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffec4a-8ad6-4384-bb55-adcb6a7fd7c2 None admin] PATH_INFO: `/v3/users/a5e74655bc0d48558528bf252f846eda` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.011910 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffec4a-8ad6-4384-bb55-adcb6a7fd7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=a5e74655bc0d48558528bf252f846eda)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:12.013399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffec4a-8ad6-4384-bb55-adcb6a7fd7c2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:12.015209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 38/74] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:11 2022] GET /identity/v3/users/a5e74655bc0d48558528bf252f846eda => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:12.016177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4976892-5770-49eb-89a7-b246d087d585 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.016995 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4976892-5770-49eb-89a7-b246d087d585 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.017266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4976892-5770-49eb-89a7-b246d087d585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.036488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c4976892-5770-49eb-89a7-b246d087d585 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:12.037687 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c4976892-5770-49eb-89a7-b246d087d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:12.071249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:12.088613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:12.091076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 37/75] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:12.096914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9369cb1-c3da-4167-ac85-6b426e79f289 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.096914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9369cb1-c3da-4167-ac85-6b426e79f289 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.096914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9369cb1-c3da-4167-ac85-6b426e79f289 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.096914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 38/76] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:12.096914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9369cb1-c3da-4167-ac85-6b426e79f289 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.110235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 39/77] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:12.111616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4976892-5770-49eb-89a7-b246d087d585 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.203652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9369cb1-c3da-4167-ac85-6b426e79f289 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:12.210176 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4976892-5770-49eb-89a7-b246d087d585 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:12.237323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0338f9e-e766-49c8-ba53-68fc0bcf9b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:12.237938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0338f9e-e766-49c8-ba53-68fc0bcf9b4d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.238210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0338f9e-e766-49c8-ba53-68fc0bcf9b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.238477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0338f9e-e766-49c8-ba53-68fc0bcf9b4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.242845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e91f0-aba7-4227-a08a-4e3ef44f42ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:12.243445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e91f0-aba7-4227-a08a-4e3ef44f42ed None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.243727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e91f0-aba7-4227-a08a-4e3ef44f42ed None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.244004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e91f0-aba7-4227-a08a-4e3ef44f42ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.248545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0338f9e-e766-49c8-ba53-68fc0bcf9b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f0338f9e-e766-49c8-ba53-68fc0bcf9b4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:12.253865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:12.254871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 39/78] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:12.261309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0338f9e-e766-49c8-ba53-68fc0bcf9b4d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.298484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:12.298828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:12.311518 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e91f0-aba7-4227-a08a-4e3ef44f42ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:12.313040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e91f0-aba7-4227-a08a-4e3ef44f42ed None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:12.354791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0338f9e-e766-49c8-ba53-68fc0bcf9b4d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:12.387442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643b145-5abf-4b33-ad08-e21197af9fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:12.388054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643b145-5abf-4b33-ad08-e21197af9fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.388334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643b145-5abf-4b33-ad08-e21197af9fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.388636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643b145-5abf-4b33-ad08-e21197af9fa0 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.389766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643b145-5abf-4b33-ad08-e21197af9fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:12.391223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643b145-5abf-4b33-ad08-e21197af9fa0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:12.393721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 40/79] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:12.394605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6d803-5426-41c6-a541-4d5d5b007fee None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.394887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6d803-5426-41c6-a541-4d5d5b007fee None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.395159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6d803-5426-41c6-a541-4d5d5b007fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.412555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c2d6d803-5426-41c6-a541-4d5d5b007fee None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:12.413747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c2d6d803-5426-41c6-a541-4d5d5b007fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:12.414005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 40/80] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:12.414837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e91f0-aba7-4227-a08a-4e3ef44f42ed None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.483623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 41/81] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:12.484538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6d803-5426-41c6-a541-4d5d5b007fee None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.499921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:12.515892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e91f0-aba7-4227-a08a-4e3ef44f42ed None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:12.527544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:12.549008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fb6b0-eecd-4426-aaaa-cf9ca3234a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:12.549626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fb6b0-eecd-4426-aaaa-cf9ca3234a28 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.549900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fb6b0-eecd-4426-aaaa-cf9ca3234a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.550167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fb6b0-eecd-4426-aaaa-cf9ca3234a28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.561886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fb6b0-eecd-4426-aaaa-cf9ca3234a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-af5fb6b0-eecd-4426-aaaa-cf9ca3234a28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:12.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:12.568111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:12.568911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 41/82] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:12.569536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fb6b0-eecd-4426-aaaa-cf9ca3234a28 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.589555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6d803-5426-41c6-a541-4d5d5b007fee None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:12.622643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30027d05-c68a-4708-87de-6350f7c0e977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:12.623175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30027d05-c68a-4708-87de-6350f7c0e977 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.623450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30027d05-c68a-4708-87de-6350f7c0e977 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.623714 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30027d05-c68a-4708-87de-6350f7c0e977 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.624375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30027d05-c68a-4708-87de-6350f7c0e977 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:12.624699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30027d05-c68a-4708-87de-6350f7c0e977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:12.626609 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30027d05-c68a-4708-87de-6350f7c0e977 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:12.634406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 42/83] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:12.635103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30027d05-c68a-4708-87de-6350f7c0e977 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.668846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fb6b0-eecd-4426-aaaa-cf9ca3234a28 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:12.702555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d886a9-e880-4c14-941b-24a694e827a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:12.703147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d886a9-e880-4c14-941b-24a694e827a4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.703419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d886a9-e880-4c14-941b-24a694e827a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.703685 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d886a9-e880-4c14-941b-24a694e827a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.704424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d886a9-e880-4c14-941b-24a694e827a4 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb, user.id=2c322d3415e942c2b44d2bbf25f886a1) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:12.704791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d886a9-e880-4c14-941b-24a694e827a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:12.706485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d886a9-e880-4c14-941b-24a694e827a4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:12.713913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 42/84] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/role_assignments?user.id=2c322d3415e942c2b44d2bbf25f886a1&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:12.714525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d886a9-e880-4c14-941b-24a694e827a4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.727153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:12.742990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30027d05-c68a-4708-87de-6350f7c0e977 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:12.745370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:12.780902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3c81a-3c25-4b3b-9188-789a4f29bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:12.781629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3c81a-3c25-4b3b-9188-789a4f29bd88 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.781991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3c81a-3c25-4b3b-9188-789a4f29bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.782317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3c81a-3c25-4b3b-9188-789a4f29bd88 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.783532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3c81a-3c25-4b3b-9188-789a4f29bd88 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:12.785047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3c81a-3c25-4b3b-9188-789a4f29bd88 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:12.792069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 43/85] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:12.792069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3c81a-3c25-4b3b-9188-789a4f29bd88 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.839781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d886a9-e880-4c14-941b-24a694e827a4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:12.878302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b62c0-094d-4954-ba95-94766b02c298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:12.878881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b62c0-094d-4954-ba95-94766b02c298 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.879138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b62c0-094d-4954-ba95-94766b02c298 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.879407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b62c0-094d-4954-ba95-94766b02c298 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.880551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b62c0-094d-4954-ba95-94766b02c298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:12.882019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b62c0-094d-4954-ba95-94766b02c298 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:12.883612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 43/86] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:12.888428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b62c0-094d-4954-ba95-94766b02c298 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.913560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3c81a-3c25-4b3b-9188-789a4f29bd88 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:12.949308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278be5f7-e39b-4793-82dc-1c3bfd59ddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:12.949932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278be5f7-e39b-4793-82dc-1c3bfd59ddba None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:12.950194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278be5f7-e39b-4793-82dc-1c3bfd59ddba None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:12.950454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278be5f7-e39b-4793-82dc-1c3bfd59ddba None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:12.951117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278be5f7-e39b-4793-82dc-1c3bfd59ddba None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:12.951419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278be5f7-e39b-4793-82dc-1c3bfd59ddba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:12.952852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278be5f7-e39b-4793-82dc-1c3bfd59ddba None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:12.961780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 44/87] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:12.966935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278be5f7-e39b-4793-82dc-1c3bfd59ddba None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:12.971601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:12.993352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b62c0-094d-4954-ba95-94766b02c298 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:13.002717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:13.028455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbc27a-980b-4216-a0e8-353f5ca0a415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:13.029352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbc27a-980b-4216-a0e8-353f5ca0a415 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:13.029481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbc27a-980b-4216-a0e8-353f5ca0a415 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:13.029712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbc27a-980b-4216-a0e8-353f5ca0a415 None admin] PATH_INFO: `/v3/users/a5e74655bc0d48558528bf252f846eda` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:13.030998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbc27a-980b-4216-a0e8-353f5ca0a415 None admin] RBAC: Authorizing `identity:get_user(user_id=a5e74655bc0d48558528bf252f846eda)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:13.032536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbc27a-980b-4216-a0e8-353f5ca0a415 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:13.038856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 44/88] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/users/a5e74655bc0d48558528bf252f846eda => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:13.038856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbc27a-980b-4216-a0e8-353f5ca0a415 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:13.084399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278be5f7-e39b-4793-82dc-1c3bfd59ddba None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:13.119925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c23364-053c-4d85-bb64-f77823555a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:13.120591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c23364-053c-4d85-bb64-f77823555a87 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:13.120816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c23364-053c-4d85-bb64-f77823555a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:13.121091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c23364-053c-4d85-bb64-f77823555a87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:13.121841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c23364-053c-4d85-bb64-f77823555a87 None admin] RBAC: Adding query filter params (role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb, user.id=a5e74655bc0d48558528bf252f846eda) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:13.122150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c23364-053c-4d85-bb64-f77823555a87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:13.123613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c23364-053c-4d85-bb64-f77823555a87 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:13.136836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 45/89] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:12 2022] GET /identity/v3/role_assignments?user.id=a5e74655bc0d48558528bf252f846eda&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:13.136836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c23364-053c-4d85-bb64-f77823555a87 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:13.150316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbc27a-980b-4216-a0e8-353f5ca0a415 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:13.192397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150e47b-6a81-4319-900b-32a6c9761e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:13.193053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150e47b-6a81-4319-900b-32a6c9761e41 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:13.193331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150e47b-6a81-4319-900b-32a6c9761e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:13.193712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150e47b-6a81-4319-900b-32a6c9761e41 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:13.194367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150e47b-6a81-4319-900b-32a6c9761e41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:13.195868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150e47b-6a81-4319-900b-32a6c9761e41 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:13.252915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:13.259711 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c23364-053c-4d85-bb64-f77823555a87 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:13.292635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 45/90] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:50:13 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:13.292635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150e47b-6a81-4319-900b-32a6c9761e41 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:13.301893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:13.310036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eedba3-aa20-4765-af66-cdca3b3b311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:13.310267 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eedba3-aa20-4765-af66-cdca3b3b311f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:13.311420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eedba3-aa20-4765-af66-cdca3b3b311f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:13.311420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eedba3-aa20-4765-af66-cdca3b3b311f None admin] PATH_INFO: `/v3/users/a5e74655bc0d48558528bf252f846eda` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:13.312218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eedba3-aa20-4765-af66-cdca3b3b311f None admin] RBAC: Authorizing `identity:get_user(user_id=a5e74655bc0d48558528bf252f846eda)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:13.314864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eedba3-aa20-4765-af66-cdca3b3b311f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:13.316045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 46/91] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:13 2022] GET /identity/v3/users/a5e74655bc0d48558528bf252f846eda => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:13.324030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eedba3-aa20-4765-af66-cdca3b3b311f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:13.442721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150e47b-6a81-4319-900b-32a6c9761e41 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:13.491986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e24c00-ca2d-468e-bf0d-665a3b85df82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:13.492676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e24c00-ca2d-468e-bf0d-665a3b85df82 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:13.492911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e24c00-ca2d-468e-bf0d-665a3b85df82 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:13.493181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e24c00-ca2d-468e-bf0d-665a3b85df82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:13.515874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eedba3-aa20-4765-af66-cdca3b3b311f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:13.567421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2fca9-2921-41b5-be7c-2b315d6bdf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:13.568369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2fca9-2921-41b5-be7c-2b315d6bdf6e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:13.568679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2fca9-2921-41b5-be7c-2b315d6bdf6e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:13.569047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2fca9-2921-41b5-be7c-2b315d6bdf6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:13.578970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e24c00-ca2d-468e-bf0d-665a3b85df82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:13.580778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e24c00-ca2d-468e-bf0d-665a3b85df82 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:13.582770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:13.612782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:13.659905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2fca9-2921-41b5-be7c-2b315d6bdf6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:13.661708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2fca9-2921-41b5-be7c-2b315d6bdf6e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:13.788029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 46/92] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:13.793905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e24c00-ca2d-468e-bf0d-665a3b85df82 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:13.814466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 47/93] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:13.821159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2fca9-2921-41b5-be7c-2b315d6bdf6e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:13.896835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:13.920958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e24c00-ca2d-468e-bf0d-665a3b85df82 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:13.923285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:13.965553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2fca9-2921-41b5-be7c-2b315d6bdf6e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:13.965818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fd5b5-7cea-4c3e-83fd-ef4db569b84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:13.966470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fd5b5-7cea-4c3e-83fd-ef4db569b84f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:13.966754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fd5b5-7cea-4c3e-83fd-ef4db569b84f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:13.967030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fd5b5-7cea-4c3e-83fd-ef4db569b84f None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:13.975280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fd5b5-7cea-4c3e-83fd-ef4db569b84f None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:13.976798 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fd5b5-7cea-4c3e-83fd-ef4db569b84f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:13.978503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 47/94] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:13 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:13.984449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fd5b5-7cea-4c3e-83fd-ef4db569b84f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:14.018458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022265e4-36a2-4d62-9de6-824c6b896836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:14.019107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022265e4-36a2-4d62-9de6-824c6b896836 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:14.019438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022265e4-36a2-4d62-9de6-824c6b896836 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:14.019712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022265e4-36a2-4d62-9de6-824c6b896836 None admin] PATH_INFO: `/v3/projects/4c7a14df803f48cfac92b8f9d9bf7c3d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:14.020891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022265e4-36a2-4d62-9de6-824c6b896836 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7a14df803f48cfac92b8f9d9bf7c3d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:14.022818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022265e4-36a2-4d62-9de6-824c6b896836 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:14.025126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 48/95] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:13 2022] GET /identity/v3/projects/4c7a14df803f48cfac92b8f9d9bf7c3d => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:14.032444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022265e4-36a2-4d62-9de6-824c6b896836 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:14.143550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022265e4-36a2-4d62-9de6-824c6b896836 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:14.176920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:14.186104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff930343-fa3b-4a45-8c4d-3bc0c1ee994b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:14.186785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff930343-fa3b-4a45-8c4d-3bc0c1ee994b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:14.187068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff930343-fa3b-4a45-8c4d-3bc0c1ee994b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:14.187339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff930343-fa3b-4a45-8c4d-3bc0c1ee994b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:14.188107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff930343-fa3b-4a45-8c4d-3bc0c1ee994b None admin] RBAC: Adding query filter params (role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=4c7a14df803f48cfac92b8f9d9bf7c3d, user.id=a5e74655bc0d48558528bf252f846eda) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:14.188419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff930343-fa3b-4a45-8c4d-3bc0c1ee994b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:14.189909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff930343-fa3b-4a45-8c4d-3bc0c1ee994b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:14.204976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 49/96] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:14 2022] GET /identity/v3/role_assignments?user.id=a5e74655bc0d48558528bf252f846eda&scope.project.id=4c7a14df803f48cfac92b8f9d9bf7c3d&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:14.210035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fd5b5-7cea-4c3e-83fd-ef4db569b84f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:14.255439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f429c-f139-49c9-ab35-1d24ede19fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:14.256040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f429c-f139-49c9-ab35-1d24ede19fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:14.256316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f429c-f139-49c9-ab35-1d24ede19fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:14.256610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f429c-f139-49c9-ab35-1d24ede19fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:14.257344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f429c-f139-49c9-ab35-1d24ede19fa0 None admin] RBAC: Adding query filter params (role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb, user.id=a5e74655bc0d48558528bf252f846eda) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:14.257646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f429c-f139-49c9-ab35-1d24ede19fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:14.259138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f429c-f139-49c9-ab35-1d24ede19fa0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:14.265766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 48/97] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:13 2022] GET /identity/v3/role_assignments?user.id=a5e74655bc0d48558528bf252f846eda&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:15.251882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f66fd-d008-4375-8c4d-c04b03d67628 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:15.252356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f66fd-d008-4375-8c4d-c04b03d67628 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:15.252356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f66fd-d008-4375-8c4d-c04b03d67628 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:15.253887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 50/98] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:15.258968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374938c-add6-4f29-bbd0-ed82891b5042 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:15.259398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374938c-add6-4f29-bbd0-ed82891b5042 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:15.259591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374938c-add6-4f29-bbd0-ed82891b5042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:15.285496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f374938c-add6-4f29-bbd0-ed82891b5042 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:15.286689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f374938c-add6-4f29-bbd0-ed82891b5042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:15.319181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:15.370758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 49/99] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:15.403200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f66fd-d008-4375-8c4d-c04b03d67628 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:15.541690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f66fd-d008-4375-8c4d-c04b03d67628 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:15.585235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ed015-01e3-4626-bd66-fb3e54c3d4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:15.585911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ed015-01e3-4626-bd66-fb3e54c3d4ea None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:15.586190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ed015-01e3-4626-bd66-fb3e54c3d4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:15.586513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ed015-01e3-4626-bd66-fb3e54c3d4ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:15.599734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ed015-01e3-4626-bd66-fb3e54c3d4ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-137ed015-01e3-4626-bd66-fb3e54c3d4ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:15.605025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:15.605888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:15.606653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 51/100] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:15.613863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374938c-add6-4f29-bbd0-ed82891b5042 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:15.756285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374938c-add6-4f29-bbd0-ed82891b5042 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:15.796213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10dd2-0dbb-4532-b867-04fff8d5e5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:15.796778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10dd2-0dbb-4532-b867-04fff8d5e5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:15.797055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10dd2-0dbb-4532-b867-04fff8d5e5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:15.797858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10dd2-0dbb-4532-b867-04fff8d5e5a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:15.801543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10dd2-0dbb-4532-b867-04fff8d5e5a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:15.801543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10dd2-0dbb-4532-b867-04fff8d5e5a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:15.801543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10dd2-0dbb-4532-b867-04fff8d5e5a0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:15.809677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 50/101] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:15.815640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ed015-01e3-4626-bd66-fb3e54c3d4ea None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:15.918443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:15.970985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ed015-01e3-4626-bd66-fb3e54c3d4ea None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:16.005605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6502e9e-ced7-4c32-bd92-e0ac24c6fb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:16.006240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502e9e-ced7-4c32-bd92-e0ac24c6fb83 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:16.006516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502e9e-ced7-4c32-bd92-e0ac24c6fb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:16.006784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502e9e-ced7-4c32-bd92-e0ac24c6fb83 None admin] PATH_INFO: `/v3/users/2c322d3415e942c2b44d2bbf25f886a1` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:16.007930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6502e9e-ced7-4c32-bd92-e0ac24c6fb83 None admin] RBAC: Authorizing `identity:get_user(user_id=2c322d3415e942c2b44d2bbf25f886a1)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:16.009437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6502e9e-ced7-4c32-bd92-e0ac24c6fb83 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:16.011234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 52/102] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:15 2022] GET /identity/v3/users/2c322d3415e942c2b44d2bbf25f886a1 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:16.016799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10dd2-0dbb-4532-b867-04fff8d5e5a0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:16.037372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829c4fe-6079-4a74-888e-8961804788c4 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:16.037687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829c4fe-6079-4a74-888e-8961804788c4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:16.037914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829c4fe-6079-4a74-888e-8961804788c4 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:16.039246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 53/103] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:16.044912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e924357-995d-4b18-8c9b-b8368be68d08 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:16.045187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e924357-995d-4b18-8c9b-b8368be68d08 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:16.045461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e924357-995d-4b18-8c9b-b8368be68d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:16.053564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:16.078739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8e924357-995d-4b18-8c9b-b8368be68d08 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:16.079955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8e924357-995d-4b18-8c9b-b8368be68d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:16.218742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10dd2-0dbb-4532-b867-04fff8d5e5a0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:16.235455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 54/104] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:16.277585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e924357-995d-4b18-8c9b-b8368be68d08 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:16.294469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3cd8f-86a6-4d6f-9ec4-cf844049c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:16.295730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3cd8f-86a6-4d6f-9ec4-cf844049c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:16.296206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3cd8f-86a6-4d6f-9ec4-cf844049c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:16.296698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3cd8f-86a6-4d6f-9ec4-cf844049c3f4 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:16.299070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3cd8f-86a6-4d6f-9ec4-cf844049c3f4 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:16.300788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3cd8f-86a6-4d6f-9ec4-cf844049c3f4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:16.302650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 51/105] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:16 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:16.311594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3cd8f-86a6-4d6f-9ec4-cf844049c3f4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:16.369427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:16.441227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e924357-995d-4b18-8c9b-b8368be68d08 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:16.488480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8538940-0303-4d44-b49b-ae2898b3184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:16.489143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8538940-0303-4d44-b49b-ae2898b3184f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:16.489421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8538940-0303-4d44-b49b-ae2898b3184f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:16.489691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8538940-0303-4d44-b49b-ae2898b3184f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:16.498904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:16.499987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8538940-0303-4d44-b49b-ae2898b3184f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c8538940-0303-4d44-b49b-ae2898b3184f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:16.505466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:16.506318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:16.507458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 55/106] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:16.508238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c747b-a81f-4395-8e7a-00291d827551 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:16.508944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c747b-a81f-4395-8e7a-00291d827551 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:16.508944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c747b-a81f-4395-8e7a-00291d827551 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:16.514108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 56/107] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:16.515424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fe9ce-c1cc-4842-8e4b-fe2075630dce None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:16.515849 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fe9ce-c1cc-4842-8e4b-fe2075630dce None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:16.515959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3cd8f-86a6-4d6f-9ec4-cf844049c3f4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:16.516397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fe9ce-c1cc-4842-8e4b-fe2075630dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:16.534814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1d3fe9ce-c1cc-4842-8e4b-fe2075630dce None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:16.536272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1d3fe9ce-c1cc-4842-8e4b-fe2075630dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:16.597095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bbdae-258e-4252-b4d9-27389001ccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:16.597732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bbdae-258e-4252-b4d9-27389001ccd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:16.598016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bbdae-258e-4252-b4d9-27389001ccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:16.598334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bbdae-258e-4252-b4d9-27389001ccd2 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/users/2c322d3415e942c2b44d2bbf25f886a1/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:16.605878 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bbdae-258e-4252-b4d9-27389001ccd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f47f695da6654f48a1df25b7f4bf15eb, user_id=2c322d3415e942c2b44d2bbf25f886a1, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:16.606806 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 57/108] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:16.607515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bbdae-258e-4252-b4d9-27389001ccd2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:16.607978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fe9ce-c1cc-4842-8e4b-fe2075630dce None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:16.688960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:16.710701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 52/109] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:50:16 2022] PUT /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/users/2c322d3415e942c2b44d2bbf25f886a1/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:16.711545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bbdae-258e-4252-b4d9-27389001ccd2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:16.736318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fe9ce-c1cc-4842-8e4b-fe2075630dce None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:16.775465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b18279-62ff-47d6-a034-826d71cb2ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:16.776462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b18279-62ff-47d6-a034-826d71cb2ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:16.776964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b18279-62ff-47d6-a034-826d71cb2ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:16.777508 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b18279-62ff-47d6-a034-826d71cb2ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:16.778398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b18279-62ff-47d6-a034-826d71cb2ce4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:16.778906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b18279-62ff-47d6-a034-826d71cb2ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:16.780498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b18279-62ff-47d6-a034-826d71cb2ce4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:16.790093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 58/110] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:16.798844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b18279-62ff-47d6-a034-826d71cb2ce4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:16.927821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bbdae-258e-4252-b4d9-27389001ccd2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:17.000160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b69e4e-ce6d-417b-97e3-724b36b26dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:17.001228 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b69e4e-ce6d-417b-97e3-724b36b26dbd None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.001753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b69e4e-ce6d-417b-97e3-724b36b26dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.002378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b69e4e-ce6d-417b-97e3-724b36b26dbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.010500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b18279-62ff-47d6-a034-826d71cb2ce4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:17.015398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b69e4e-ce6d-417b-97e3-724b36b26dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-46b69e4e-ce6d-417b-97e3-724b36b26dbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:17.023434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:17.024362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:17.025160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:17.025742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:17.028005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 53/111] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:17.035380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b69e4e-ce6d-417b-97e3-724b36b26dbd None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:17.060551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:17.089998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281c065-70e6-4f36-bbab-df16f034e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:17.090633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281c065-70e6-4f36-bbab-df16f034e4fc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.090906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281c065-70e6-4f36-bbab-df16f034e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.091173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281c065-70e6-4f36-bbab-df16f034e4fc None admin] PATH_INFO: `/v3/users/a5e74655bc0d48558528bf252f846eda` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.092636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281c065-70e6-4f36-bbab-df16f034e4fc None admin] RBAC: Authorizing `identity:get_user(user_id=a5e74655bc0d48558528bf252f846eda)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:17.096729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281c065-70e6-4f36-bbab-df16f034e4fc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:17.098542 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 59/112] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:16 2022] GET /identity/v3/users/a5e74655bc0d48558528bf252f846eda => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:17.102332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120676e0-8231-48a9-a8ef-c921c0377b9f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.102619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120676e0-8231-48a9-a8ef-c921c0377b9f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.102891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120676e0-8231-48a9-a8ef-c921c0377b9f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.104411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 60/113] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:17.104979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120676e0-8231-48a9-a8ef-c921c0377b9f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:17.181309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b69e4e-ce6d-417b-97e3-724b36b26dbd None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:17.225674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1378ef4-6f26-4887-bb0d-c6ea9bb03078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:17.226706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1378ef4-6f26-4887-bb0d-c6ea9bb03078 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.226991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1378ef4-6f26-4887-bb0d-c6ea9bb03078 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.227260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1378ef4-6f26-4887-bb0d-c6ea9bb03078 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.228208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1378ef4-6f26-4887-bb0d-c6ea9bb03078 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:17.228542 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1378ef4-6f26-4887-bb0d-c6ea9bb03078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:17.230089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1378ef4-6f26-4887-bb0d-c6ea9bb03078 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:17.244402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 54/114] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:17.244402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6a359-c1e9-4cba-b6ac-879f3c347eea None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.244402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6a359-c1e9-4cba-b6ac-879f3c347eea None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.244402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6a359-c1e9-4cba-b6ac-879f3c347eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.252770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120676e0-8231-48a9-a8ef-c921c0377b9f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:17.263599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6cf6a359-c1e9-4cba-b6ac-879f3c347eea None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:17.265149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6cf6a359-c1e9-4cba-b6ac-879f3c347eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:17.291832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556d80d-25dd-47d3-8e85-41257a54f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:17.293195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556d80d-25dd-47d3-8e85-41257a54f4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.293672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556d80d-25dd-47d3-8e85-41257a54f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.294267 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556d80d-25dd-47d3-8e85-41257a54f4f0 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.295948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556d80d-25dd-47d3-8e85-41257a54f4f0 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:17.298002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556d80d-25dd-47d3-8e85-41257a54f4f0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:17.300438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 61/115] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:17 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:17.306940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c22efe-beb5-4a6b-b22d-a530fac16c1a None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.306940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c22efe-beb5-4a6b-b22d-a530fac16c1a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.306940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c22efe-beb5-4a6b-b22d-a530fac16c1a None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.306940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 62/116] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:17.307261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c22efe-beb5-4a6b-b22d-a530fac16c1a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:17.360598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 55/117] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:17.362169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6a359-c1e9-4cba-b6ac-879f3c347eea None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:17.364657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:17.378798 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:17.445505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c22efe-beb5-4a6b-b22d-a530fac16c1a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:17.480232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6a359-c1e9-4cba-b6ac-879f3c347eea None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:17.483602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71da7b-cd30-4ddc-890f-10c41cd23341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:17.484223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71da7b-cd30-4ddc-890f-10c41cd23341 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.484513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71da7b-cd30-4ddc-890f-10c41cd23341 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.484835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71da7b-cd30-4ddc-890f-10c41cd23341 None admin] PATH_INFO: `/v3/users/81a7f8d5b07645229d40b5b27a52049c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.497898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71da7b-cd30-4ddc-890f-10c41cd23341 None admin] RBAC: Authorizing `identity:get_user(user_id=81a7f8d5b07645229d40b5b27a52049c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:17.499368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71da7b-cd30-4ddc-890f-10c41cd23341 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:17.501770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 63/118] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:17 2022] GET /identity/v3/users/81a7f8d5b07645229d40b5b27a52049c => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:17.502785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad97c53-fdc6-463a-b937-de7c8e93aff6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.503064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad97c53-fdc6-463a-b937-de7c8e93aff6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.503329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad97c53-fdc6-463a-b937-de7c8e93aff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.521098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9ad97c53-fdc6-463a-b937-de7c8e93aff6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:17.522093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457e571-486b-482f-a14d-5ac8c43436bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:17.522319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9ad97c53-fdc6-463a-b937-de7c8e93aff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:17.522752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457e571-486b-482f-a14d-5ac8c43436bc None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.523029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457e571-486b-482f-a14d-5ac8c43436bc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.523298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457e571-486b-482f-a14d-5ac8c43436bc None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/users/a5e74655bc0d48558528bf252f846eda/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.525475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457e571-486b-482f-a14d-5ac8c43436bc None admin] RBAC: Authorizing `identity:create_grant(project_id=f47f695da6654f48a1df25b7f4bf15eb, user_id=a5e74655bc0d48558528bf252f846eda, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:17.526917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457e571-486b-482f-a14d-5ac8c43436bc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:17.587828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 64/119] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:17.589383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad97c53-fdc6-463a-b937-de7c8e93aff6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:17.615993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 56/120] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:50:17 2022] PUT /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/users/a5e74655bc0d48558528bf252f846eda/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:17.617225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28c6fa-b13f-4ca7-adf0-b8ba0581f355 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.617673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28c6fa-b13f-4ca7-adf0-b8ba0581f355 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.617962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28c6fa-b13f-4ca7-adf0-b8ba0581f355 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.623695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 57/121] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:17.623695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28c6fa-b13f-4ca7-adf0-b8ba0581f355 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:17.634816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:17.737235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:17.795747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad97c53-fdc6-463a-b937-de7c8e93aff6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:17.800537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28c6fa-b13f-4ca7-adf0-b8ba0581f355 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:17.837540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc3f9-664b-4a48-ae8c-2cfcf3feac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:17.838173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc3f9-664b-4a48-ae8c-2cfcf3feac45 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.838449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc3f9-664b-4a48-ae8c-2cfcf3feac45 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.838722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc3f9-664b-4a48-ae8c-2cfcf3feac45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.851146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcc3f9-664b-4a48-ae8c-2cfcf3feac45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c9bcc3f9-664b-4a48-ae8c-2cfcf3feac45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:17.856928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:17.858483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 65/122] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:17.861540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc3f9-664b-4a48-ae8c-2cfcf3feac45 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:17.869511 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8abc-de3f-4f61-826e-dd819a1d4e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:17.870906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8abc-de3f-4f61-826e-dd819a1d4e34 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:17.871638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8abc-de3f-4f61-826e-dd819a1d4e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:17.872481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8abc-de3f-4f61-826e-dd819a1d4e34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:17.974606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:17.986650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f8abc-de3f-4f61-826e-dd819a1d4e34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:17.987133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc3f9-664b-4a48-ae8c-2cfcf3feac45 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:17.988727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f8abc-de3f-4f61-826e-dd819a1d4e34 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:18.029005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacb3f6-0473-4294-a6f6-0df56016b902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:18.029993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacb3f6-0473-4294-a6f6-0df56016b902 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:18.030494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacb3f6-0473-4294-a6f6-0df56016b902 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:18.030963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacb3f6-0473-4294-a6f6-0df56016b902 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:18.032363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eacb3f6-0473-4294-a6f6-0df56016b902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:18.034059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eacb3f6-0473-4294-a6f6-0df56016b902 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:18.036174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 66/123] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:18.037656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e57d86-d196-4c7b-8f88-724926449844 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:18.038145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e57d86-d196-4c7b-8f88-724926449844 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:18.038614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e57d86-d196-4c7b-8f88-724926449844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:18.058116 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c6e57d86-d196-4c7b-8f88-724926449844 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:18.059654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c6e57d86-d196-4c7b-8f88-724926449844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:18.066529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:18.108560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 58/124] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:18.113641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8abc-de3f-4f61-826e-dd819a1d4e34 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:18.140660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 67/125] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:18.142077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e57d86-d196-4c7b-8f88-724926449844 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:18.227193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8abc-de3f-4f61-826e-dd819a1d4e34 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:18.250859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:18.263477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e57d86-d196-4c7b-8f88-724926449844 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:18.264994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db1e48-90c3-4e5a-ad6d-f90901058af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:18.265946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1e48-90c3-4e5a-ad6d-f90901058af4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:18.266370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1e48-90c3-4e5a-ad6d-f90901058af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:18.266810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1e48-90c3-4e5a-ad6d-f90901058af4 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:18.267652 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db1e48-90c3-4e5a-ad6d-f90901058af4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:18.268092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db1e48-90c3-4e5a-ad6d-f90901058af4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:18.269662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db1e48-90c3-4e5a-ad6d-f90901058af4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:18.278437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 59/126] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:18.280534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db1e48-90c3-4e5a-ad6d-f90901058af4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:18.309134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200055bf-2f94-4d80-b65d-cf43ad19827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:18.309899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200055bf-2f94-4d80-b65d-cf43ad19827e None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:18.310277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200055bf-2f94-4d80-b65d-cf43ad19827e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:18.310777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200055bf-2f94-4d80-b65d-cf43ad19827e None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:18.311527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200055bf-2f94-4d80-b65d-cf43ad19827e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:18.313298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200055bf-2f94-4d80-b65d-cf43ad19827e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:18.314245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:18.389969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db1e48-90c3-4e5a-ad6d-f90901058af4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:18.408426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 68/127] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:50:18 2022] POST /identity/v3/projects => generated 302 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:18.412568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200055bf-2f94-4d80-b65d-cf43ad19827e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:18.436009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94215f8-b477-43dc-8384-2b030d2e6488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:18.436680 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94215f8-b477-43dc-8384-2b030d2e6488 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:18.436961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94215f8-b477-43dc-8384-2b030d2e6488 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:18.437254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94215f8-b477-43dc-8384-2b030d2e6488 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:18.438403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94215f8-b477-43dc-8384-2b030d2e6488 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:18.439914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94215f8-b477-43dc-8384-2b030d2e6488 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:18.441674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 60/128] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:18 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:18.442468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94215f8-b477-43dc-8384-2b030d2e6488 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:18.572155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200055bf-2f94-4d80-b65d-cf43ad19827e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:18.621598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:18.623729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f1014-c429-4f3c-8b53-d6063470b45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:18.624364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1014-c429-4f3c-8b53-d6063470b45c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:18.624840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1014-c429-4f3c-8b53-d6063470b45c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:18.625116 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1014-c429-4f3c-8b53-d6063470b45c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:18.630816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94215f8-b477-43dc-8384-2b030d2e6488 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:18.635742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:18.639557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f1014-c429-4f3c-8b53-d6063470b45c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2f5f1014-c429-4f3c-8b53-d6063470b45c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:18.644946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:18.645812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:18.646600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:18.650981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 69/129] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:18.651900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f1014-c429-4f3c-8b53-d6063470b45c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:18.675566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfc22f-c615-48d0-b164-6073607394a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:18.676671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfc22f-c615-48d0-b164-6073607394a9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:18.678434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfc22f-c615-48d0-b164-6073607394a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:18.679161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfc22f-c615-48d0-b164-6073607394a9 None admin] PATH_INFO: `/v3/users/a5e74655bc0d48558528bf252f846eda` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:18.680707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfc22f-c615-48d0-b164-6073607394a9 None admin] RBAC: Authorizing `identity:get_user(user_id=a5e74655bc0d48558528bf252f846eda)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:18.683008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfc22f-c615-48d0-b164-6073607394a9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:18.687897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 61/130] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:18 2022] GET /identity/v3/users/a5e74655bc0d48558528bf252f846eda => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:18.687897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfc22f-c615-48d0-b164-6073607394a9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:18.796011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f1014-c429-4f3c-8b53-d6063470b45c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:18.798697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfc22f-c615-48d0-b164-6073607394a9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:18.835589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0bc31-c3c8-4531-8bab-a37a923c2a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:18.836156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0bc31-c3c8-4531-8bab-a37a923c2a20 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:18.836551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0bc31-c3c8-4531-8bab-a37a923c2a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:18.837144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0bc31-c3c8-4531-8bab-a37a923c2a20 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:18.837700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0bc31-c3c8-4531-8bab-a37a923c2a20 None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:18.838104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0bc31-c3c8-4531-8bab-a37a923c2a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:18.839570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0bc31-c3c8-4531-8bab-a37a923c2a20 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:18.842593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68cb260-04e0-45a0-9283-069920974878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:18.843361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cb260-04e0-45a0-9283-069920974878 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:18.843775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cb260-04e0-45a0-9283-069920974878 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:18.844780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cb260-04e0-45a0-9283-069920974878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:18.845799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68cb260-04e0-45a0-9283-069920974878 None admin] RBAC: Adding query filter params (role.id=17973d7fca9143289f3ca113a29e4166, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb, user.id=81a7f8d5b07645229d40b5b27a52049c) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:18.846238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68cb260-04e0-45a0-9283-069920974878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:18.847838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68cb260-04e0-45a0-9283-069920974878 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:18.850535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 62/131] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:18.854854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0bc31-c3c8-4531-8bab-a37a923c2a20 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:18.859403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 70/132] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:18 2022] GET /identity/v3/role_assignments?user.id=81a7f8d5b07645229d40b5b27a52049c&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=17973d7fca9143289f3ca113a29e4166 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:18.861407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68cb260-04e0-45a0-9283-069920974878 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:18.885175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:18.934448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:18.963502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0bc31-c3c8-4531-8bab-a37a923c2a20 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:18.997153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68cb260-04e0-45a0-9283-069920974878 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:19.005520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6043fa-0bb5-4b59-88ab-7efdab3177cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:19.006303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6043fa-0bb5-4b59-88ab-7efdab3177cd None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:19.006671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6043fa-0bb5-4b59-88ab-7efdab3177cd None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:19.007110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6043fa-0bb5-4b59-88ab-7efdab3177cd None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:19.008335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6043fa-0bb5-4b59-88ab-7efdab3177cd None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:19.010018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6043fa-0bb5-4b59-88ab-7efdab3177cd None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:19.012549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 63/133] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:18 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:19.018068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6043fa-0bb5-4b59-88ab-7efdab3177cd None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:19.070903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26d3d9-aec9-40cd-9aaf-e0e9a617a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:19.071912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26d3d9-aec9-40cd-9aaf-e0e9a617a85c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:19.072376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26d3d9-aec9-40cd-9aaf-e0e9a617a85c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:19.072860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26d3d9-aec9-40cd-9aaf-e0e9a617a85c None admin] PATH_INFO: `/v3/users/a5e74655bc0d48558528bf252f846eda` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:19.075087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26d3d9-aec9-40cd-9aaf-e0e9a617a85c None admin] RBAC: Authorizing `identity:get_user(user_id=a5e74655bc0d48558528bf252f846eda)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:19.076761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26d3d9-aec9-40cd-9aaf-e0e9a617a85c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:19.079934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 71/134] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:18 2022] GET /identity/v3/users/a5e74655bc0d48558528bf252f846eda => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:19.087351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26d3d9-aec9-40cd-9aaf-e0e9a617a85c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:19.223117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:19.242710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6043fa-0bb5-4b59-88ab-7efdab3177cd None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:19.248534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26d3d9-aec9-40cd-9aaf-e0e9a617a85c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:19.280388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:19.289298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c710b67-ddf7-4fad-9410-5c57e75886cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:19.291511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c710b67-ddf7-4fad-9410-5c57e75886cf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:19.292696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c710b67-ddf7-4fad-9410-5c57e75886cf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:19.292974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c710b67-ddf7-4fad-9410-5c57e75886cf None admin] PATH_INFO: `/v3/projects/4c7a14df803f48cfac92b8f9d9bf7c3d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:19.295173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c710b67-ddf7-4fad-9410-5c57e75886cf None admin] RBAC: Authorizing `identity:get_project(project_id=4c7a14df803f48cfac92b8f9d9bf7c3d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:19.296660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c710b67-ddf7-4fad-9410-5c57e75886cf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:19.298826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 72/135] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:19 2022] GET /identity/v3/projects/4c7a14df803f48cfac92b8f9d9bf7c3d => generated 312 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:19.301653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f74ff-a90c-40aa-b35b-ef63b4305622 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:19.301940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f74ff-a90c-40aa-b35b-ef63b4305622 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:19.302210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f74ff-a90c-40aa-b35b-ef63b4305622 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:19.303617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 73/136] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:19.307953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f74ff-a90c-40aa-b35b-ef63b4305622 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:19.322241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507296a-42b3-43ad-b9ce-4510d56619b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:19.322897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507296a-42b3-43ad-b9ce-4510d56619b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:19.323170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507296a-42b3-43ad-b9ce-4510d56619b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:19.323437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507296a-42b3-43ad-b9ce-4510d56619b8 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/users/a5e74655bc0d48558528bf252f846eda/roles/b2ded61baed84402a2fa33eda9462c6f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:19.326185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507296a-42b3-43ad-b9ce-4510d56619b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f47f695da6654f48a1df25b7f4bf15eb, user_id=a5e74655bc0d48558528bf252f846eda, role_id=b2ded61baed84402a2fa33eda9462c6f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:19.328097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507296a-42b3-43ad-b9ce-4510d56619b8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:19.417726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 64/137] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:50:19 2022] PUT /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/users/a5e74655bc0d48558528bf252f846eda/roles/b2ded61baed84402a2fa33eda9462c6f => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:50:19.418664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee24b4-2083-4947-9cb3-5c1c45ecc13b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:19.418993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee24b4-2083-4947-9cb3-5c1c45ecc13b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:19.419267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee24b4-2083-4947-9cb3-5c1c45ecc13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:19.462785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e0ee24b4-2083-4947-9cb3-5c1c45ecc13b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:19.465791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e0ee24b4-2083-4947-9cb3-5c1c45ecc13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:19.469857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f74ff-a90c-40aa-b35b-ef63b4305622 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:19.550212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1419f3c-2fa8-436d-a02d-334ef2960c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:19.551232 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1419f3c-2fa8-436d-a02d-334ef2960c26 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:19.551703 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1419f3c-2fa8-436d-a02d-334ef2960c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:19.552217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1419f3c-2fa8-436d-a02d-334ef2960c26 None admin] PATH_INFO: `/v3/projects/4c7a14df803f48cfac92b8f9d9bf7c3d/users/a5e74655bc0d48558528bf252f846eda/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:19.558822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1419f3c-2fa8-436d-a02d-334ef2960c26 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c7a14df803f48cfac92b8f9d9bf7c3d, user_id=a5e74655bc0d48558528bf252f846eda, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:19.560663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1419f3c-2fa8-436d-a02d-334ef2960c26 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:19.583262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 65/138] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:19.607879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee24b4-2083-4947-9cb3-5c1c45ecc13b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:19.700330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:19.718291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee24b4-2083-4947-9cb3-5c1c45ecc13b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:19.736536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 74/139] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:50:19 2022] PUT /identity/v3/projects/4c7a14df803f48cfac92b8f9d9bf7c3d/users/a5e74655bc0d48558528bf252f846eda/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:19.759239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da07ad-81f1-43e2-b844-52906dcf8651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:19.759922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da07ad-81f1-43e2-b844-52906dcf8651 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:19.760175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da07ad-81f1-43e2-b844-52906dcf8651 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:19.760438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da07ad-81f1-43e2-b844-52906dcf8651 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:19.773159 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da07ad-81f1-43e2-b844-52906dcf8651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-43da07ad-81f1-43e2-b844-52906dcf8651 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:19.778315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:19.779248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:19.780444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 66/140] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:19.788311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1419f3c-2fa8-436d-a02d-334ef2960c26 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:19.882867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:19.934304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1419f3c-2fa8-436d-a02d-334ef2960c26 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:19.973222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402e999-ec7a-4651-8739-f5a29b10fc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:19.973752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e999-ec7a-4651-8739-f5a29b10fc84 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:19.974024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e999-ec7a-4651-8739-f5a29b10fc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:19.974287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e999-ec7a-4651-8739-f5a29b10fc84 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:19.974947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e999-ec7a-4651-8739-f5a29b10fc84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:19.975240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e999-ec7a-4651-8739-f5a29b10fc84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:19.976637 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e999-ec7a-4651-8739-f5a29b10fc84 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:19.988210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 75/141] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:19.993894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da07ad-81f1-43e2-b844-52906dcf8651 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:20.034501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfd45e-ba49-4146-8a80-748d25fa71ee None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:20.034810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfd45e-ba49-4146-8a80-748d25fa71ee None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:20.035135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfd45e-ba49-4146-8a80-748d25fa71ee None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:20.036684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 76/142] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:20.042054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b16de-5807-44d7-ab38-809c7f590803 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:20.042572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b16de-5807-44d7-ab38-809c7f590803 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:20.043042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b16de-5807-44d7-ab38-809c7f590803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:20.067091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-430b16de-5807-44d7-ab38-809c7f590803 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:20.068736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-430b16de-5807-44d7-ab38-809c7f590803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:20.135418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da07ad-81f1-43e2-b844-52906dcf8651 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:20.172906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ea22f-bbad-430b-b67b-8ac571da901b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:20.173921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ea22f-bbad-430b-b67b-8ac571da901b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:20.174395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ea22f-bbad-430b-b67b-8ac571da901b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:20.174906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ea22f-bbad-430b-b67b-8ac571da901b None admin] PATH_INFO: `/v3/users/2c322d3415e942c2b44d2bbf25f886a1` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:20.176252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ea22f-bbad-430b-b67b-8ac571da901b None admin] RBAC: Authorizing `identity:get_user(user_id=2c322d3415e942c2b44d2bbf25f886a1)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:20.177943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ea22f-bbad-430b-b67b-8ac571da901b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:20.180861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 67/143] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:19 2022] GET /identity/v3/users/2c322d3415e942c2b44d2bbf25f886a1 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:20.185662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ea22f-bbad-430b-b67b-8ac571da901b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:20.206134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:20.238223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 77/144] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:20.312101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ea22f-bbad-430b-b67b-8ac571da901b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:20.325203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b16de-5807-44d7-ab38-809c7f590803 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:20.349017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c288e5-252a-4af2-8c04-77c0b67fb8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:20.349623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c288e5-252a-4af2-8c04-77c0b67fb8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:20.349950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c288e5-252a-4af2-8c04-77c0b67fb8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:20.350275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c288e5-252a-4af2-8c04-77c0b67fb8b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:20.425563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c288e5-252a-4af2-8c04-77c0b67fb8b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:20.427050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c288e5-252a-4af2-8c04-77c0b67fb8b3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:20.429629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:20.444313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:20.523953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b16de-5807-44d7-ab38-809c7f590803 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:20.535964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 68/145] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:20.541105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c288e5-252a-4af2-8c04-77c0b67fb8b3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:20.590176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752f8ed-4cca-4391-b8e5-d5d9d97447af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:20.590839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752f8ed-4cca-4391-b8e5-d5d9d97447af None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:20.591407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752f8ed-4cca-4391-b8e5-d5d9d97447af None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:20.591512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752f8ed-4cca-4391-b8e5-d5d9d97447af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:20.612278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752f8ed-4cca-4391-b8e5-d5d9d97447af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2752f8ed-4cca-4391-b8e5-d5d9d97447af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:20.617257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:20.618124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:20.618903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 78/146] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:20.626752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752f8ed-4cca-4391-b8e5-d5d9d97447af None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:20.652348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c288e5-252a-4af2-8c04-77c0b67fb8b3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:20.671036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:20.689810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f94836-5c91-446a-879c-c93da454885e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:20.690657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f94836-5c91-446a-879c-c93da454885e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:20.691045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f94836-5c91-446a-879c-c93da454885e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:20.691531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f94836-5c91-446a-879c-c93da454885e None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:20.692941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f94836-5c91-446a-879c-c93da454885e None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:20.694703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f94836-5c91-446a-879c-c93da454885e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:20.696579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 69/147] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:20 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:20.703233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f94836-5c91-446a-879c-c93da454885e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:20.808740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f94836-5c91-446a-879c-c93da454885e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:20.845422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a38ec5-275c-4034-b16f-36298ec99a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:20.846257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a38ec5-275c-4034-b16f-36298ec99a72 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:20.846632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a38ec5-275c-4034-b16f-36298ec99a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:20.847048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a38ec5-275c-4034-b16f-36298ec99a72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:20.848031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a38ec5-275c-4034-b16f-36298ec99a72 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb, user.id=2c322d3415e942c2b44d2bbf25f886a1) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:20.848444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a38ec5-275c-4034-b16f-36298ec99a72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:20.850124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a38ec5-275c-4034-b16f-36298ec99a72 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:20.856297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 70/148] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:20 2022] GET /identity/v3/role_assignments?user.id=2c322d3415e942c2b44d2bbf25f886a1&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:20.860614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752f8ed-4cca-4391-b8e5-d5d9d97447af None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:20.884872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:20.927457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b5469-158b-45d8-98a2-e58d8b6a7035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:20.928418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b5469-158b-45d8-98a2-e58d8b6a7035 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:20.928946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b5469-158b-45d8-98a2-e58d8b6a7035 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:20.929500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b5469-158b-45d8-98a2-e58d8b6a7035 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:20.930370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b5469-158b-45d8-98a2-e58d8b6a7035 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:20.930866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b5469-158b-45d8-98a2-e58d8b6a7035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:20.932438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b5469-158b-45d8-98a2-e58d8b6a7035 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:20.945017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 79/149] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:20.950559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a38ec5-275c-4034-b16f-36298ec99a72 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:21.008893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:21.076536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a38ec5-275c-4034-b16f-36298ec99a72 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:21.117282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca7052-149b-415f-9eae-ca463f7fbb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:21.118113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca7052-149b-415f-9eae-ca463f7fbb19 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:21.118498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca7052-149b-415f-9eae-ca463f7fbb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:21.118872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca7052-149b-415f-9eae-ca463f7fbb19 None admin] PATH_INFO: `/v3/users/a5e74655bc0d48558528bf252f846eda` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:21.120166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca7052-149b-415f-9eae-ca463f7fbb19 None admin] RBAC: Authorizing `identity:get_user(user_id=a5e74655bc0d48558528bf252f846eda)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:21.121833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca7052-149b-415f-9eae-ca463f7fbb19 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:21.123553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 71/150] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:20 2022] GET /identity/v3/users/a5e74655bc0d48558528bf252f846eda => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:21.129942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b5469-158b-45d8-98a2-e58d8b6a7035 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:21.247786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b5469-158b-45d8-98a2-e58d8b6a7035 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:21.282806 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016ba24-7825-4fc2-97c1-79a6d01902ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:21.283559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016ba24-7825-4fc2-97c1-79a6d01902ed None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:21.283938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016ba24-7825-4fc2-97c1-79a6d01902ed None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:21.284307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016ba24-7825-4fc2-97c1-79a6d01902ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:21.333125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:21.361098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016ba24-7825-4fc2-97c1-79a6d01902ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:21.362636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016ba24-7825-4fc2-97c1-79a6d01902ed None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:21.476263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 80/151] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:21.482393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca7052-149b-415f-9eae-ca463f7fbb19 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:21.590292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca7052-149b-415f-9eae-ca463f7fbb19 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:21.604914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:21.626594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7dc90-718c-4eab-9567-f991c29483ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:21.627258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7dc90-718c-4eab-9567-f991c29483ea None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:21.627535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7dc90-718c-4eab-9567-f991c29483ea None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:21.627806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7dc90-718c-4eab-9567-f991c29483ea None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:21.629015 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7dc90-718c-4eab-9567-f991c29483ea None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:21.630563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7dc90-718c-4eab-9567-f991c29483ea None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:21.632189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 72/152] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:21 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:21.638353 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016ba24-7825-4fc2-97c1-79a6d01902ed None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:21.734471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:21.755576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56270294-2935-40ac-ac40-c74c40fa26fc None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:21.755892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56270294-2935-40ac-ac40-c74c40fa26fc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:21.756139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56270294-2935-40ac-ac40-c74c40fa26fc None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:21.757785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 73/153] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:21.767884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f417e-dde1-4713-ae9c-71844e14dcf9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:21.768266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f417e-dde1-4713-ae9c-71844e14dcf9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:21.771341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f417e-dde1-4713-ae9c-71844e14dcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:21.771729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016ba24-7825-4fc2-97c1-79a6d01902ed None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:21.789809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9f8f417e-dde1-4713-ae9c-71844e14dcf9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:21.789809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9f8f417e-dde1-4713-ae9c-71844e14dcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:21.809274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7b507-f415-4e4e-848e-2069cc8b2c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:21.810018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7b507-f415-4e4e-848e-2069cc8b2c99 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:21.810410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7b507-f415-4e4e-848e-2069cc8b2c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:21.811365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7b507-f415-4e4e-848e-2069cc8b2c99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:21.812487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7b507-f415-4e4e-848e-2069cc8b2c99 None admin] RBAC: Adding query filter params (role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb, user.id=a5e74655bc0d48558528bf252f846eda) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:21.812953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7b507-f415-4e4e-848e-2069cc8b2c99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:21.815314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7b507-f415-4e4e-848e-2069cc8b2c99 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:21.823071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 81/154] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:21 2022] GET /identity/v3/role_assignments?user.id=a5e74655bc0d48558528bf252f846eda&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:21.824297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c122016-7dc5-443d-a996-6ff0e272b9a9 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:21.824666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c122016-7dc5-443d-a996-6ff0e272b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:21.824990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c122016-7dc5-443d-a996-6ff0e272b9a9 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:21.826483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 82/155] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:21.833221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b8ef8-5802-4016-acd7-705af311cfa2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:21.834611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b8ef8-5802-4016-acd7-705af311cfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:21.834964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b8ef8-5802-4016-acd7-705af311cfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:21.855994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ea3b8ef8-5802-4016-acd7-705af311cfa2 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:21.857870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ea3b8ef8-5802-4016-acd7-705af311cfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:21.917208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 74/156] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:21.936404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 83/157] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:21.960369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f417e-dde1-4713-ae9c-71844e14dcf9 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:21.962069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b8ef8-5802-4016-acd7-705af311cfa2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:22.079422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:22.114458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f417e-dde1-4713-ae9c-71844e14dcf9 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:22.122996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b8ef8-5802-4016-acd7-705af311cfa2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:22.126536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:22.159824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab82e1-ea01-4d6e-8a71-8b7c83009440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:22.160454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab82e1-ea01-4d6e-8a71-8b7c83009440 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.160758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab82e1-ea01-4d6e-8a71-8b7c83009440 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.161027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab82e1-ea01-4d6e-8a71-8b7c83009440 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.170449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569ba54-6c47-4c63-adfa-32a865e49f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:22.171082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569ba54-6c47-4c63-adfa-32a865e49f4a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.171353 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569ba54-6c47-4c63-adfa-32a865e49f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.171617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569ba54-6c47-4c63-adfa-32a865e49f4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.173002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569ba54-6c47-4c63-adfa-32a865e49f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:22.174478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569ba54-6c47-4c63-adfa-32a865e49f4a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:22.176086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 84/158] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:22.176665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab82e1-ea01-4d6e-8a71-8b7c83009440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ffab82e1-ea01-4d6e-8a71-8b7c83009440 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:22.181607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:22.182459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:22.183238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 75/159] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:22.183937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569ba54-6c47-4c63-adfa-32a865e49f4a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:22.191874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab82e1-ea01-4d6e-8a71-8b7c83009440 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:22.323149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab82e1-ea01-4d6e-8a71-8b7c83009440 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:22.334969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569ba54-6c47-4c63-adfa-32a865e49f4a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:22.364456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170baf6-cf4b-4d81-ac9f-f4afd9fb950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:22.365477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170baf6-cf4b-4d81-ac9f-f4afd9fb950e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.365950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170baf6-cf4b-4d81-ac9f-f4afd9fb950e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.366467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170baf6-cf4b-4d81-ac9f-f4afd9fb950e None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.367379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170baf6-cf4b-4d81-ac9f-f4afd9fb950e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:22.367897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170baf6-cf4b-4d81-ac9f-f4afd9fb950e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:22.369525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170baf6-cf4b-4d81-ac9f-f4afd9fb950e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:22.383835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946998a1-2aa2-4718-8dbe-e00eacb3842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:22.383835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946998a1-2aa2-4718-8dbe-e00eacb3842c None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.383835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946998a1-2aa2-4718-8dbe-e00eacb3842c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.383835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946998a1-2aa2-4718-8dbe-e00eacb3842c None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.384208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946998a1-2aa2-4718-8dbe-e00eacb3842c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:22.384444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946998a1-2aa2-4718-8dbe-e00eacb3842c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:22.387029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 76/160] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:22.387757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170baf6-cf4b-4d81-ac9f-f4afd9fb950e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:22.397071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:22.427136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:22.498726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170baf6-cf4b-4d81-ac9f-f4afd9fb950e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:22.518994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 85/161] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:50:22 2022] POST /identity/v3/users => generated 283 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:22.551349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7204-552b-48d2-aba1-ee8edeafc846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:22.552047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7204-552b-48d2-aba1-ee8edeafc846 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.552328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7204-552b-48d2-aba1-ee8edeafc846 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.552713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7204-552b-48d2-aba1-ee8edeafc846 None admin] PATH_INFO: `/v3/users/81a7f8d5b07645229d40b5b27a52049c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.553916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7204-552b-48d2-aba1-ee8edeafc846 None admin] RBAC: Authorizing `identity:get_user(user_id=81a7f8d5b07645229d40b5b27a52049c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:22.555540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a7204-552b-48d2-aba1-ee8edeafc846 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:22.557597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 77/162] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:22 2022] GET /identity/v3/users/81a7f8d5b07645229d40b5b27a52049c => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:22.564030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946998a1-2aa2-4718-8dbe-e00eacb3842c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:22.627942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070139b-d3a2-4884-9224-4c44abf47f10 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.628226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070139b-d3a2-4884-9224-4c44abf47f10 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.628480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070139b-d3a2-4884-9224-4c44abf47f10 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.630017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 78/163] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:22.637045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307b7f-35f6-49fd-b76a-94d588992177 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.637400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307b7f-35f6-49fd-b76a-94d588992177 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.637741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307b7f-35f6-49fd-b76a-94d588992177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.658650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2f307b7f-35f6-49fd-b76a-94d588992177 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:22.659954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2f307b7f-35f6-49fd-b76a-94d588992177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:22.689913 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946998a1-2aa2-4718-8dbe-e00eacb3842c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:22.734395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2845db-2600-4afa-8e92-51a436039411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:22.735580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2845db-2600-4afa-8e92-51a436039411 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.735850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2845db-2600-4afa-8e92-51a436039411 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.736113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2845db-2600-4afa-8e92-51a436039411 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.737249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2845db-2600-4afa-8e92-51a436039411 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:22.738693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2845db-2600-4afa-8e92-51a436039411 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:22.738797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 79/164] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:22.744600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02cde-aaac-43fd-b377-cecd273a88cc None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.744600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02cde-aaac-43fd-b377-cecd273a88cc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.744868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 86/165] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:22 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:22.744940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02cde-aaac-43fd-b377-cecd273a88cc None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.746355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 80/166] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:22.746660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2845db-2600-4afa-8e92-51a436039411 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:22.752632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7be41f-2cea-4620-934b-f6e3679fb84c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.753126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7be41f-2cea-4620-934b-f6e3679fb84c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.753395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7be41f-2cea-4620-934b-f6e3679fb84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.773326 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cb7be41f-2cea-4620-934b-f6e3679fb84c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:22.774501 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cb7be41f-2cea-4620-934b-f6e3679fb84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:22.805836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:22.815357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:22.868359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 81/167] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:22.869161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7be41f-2cea-4620-934b-f6e3679fb84c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:22.871569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2845db-2600-4afa-8e92-51a436039411 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:22.914772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffa836-1970-4b46-a1ec-9b169638189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:22.915770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffa836-1970-4b46-a1ec-9b169638189c None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:22.916258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffa836-1970-4b46-a1ec-9b169638189c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:22.916702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffa836-1970-4b46-a1ec-9b169638189c None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/users/81a7f8d5b07645229d40b5b27a52049c/roles/17973d7fca9143289f3ca113a29e4166` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:22.918816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffa836-1970-4b46-a1ec-9b169638189c None admin] RBAC: Authorizing `identity:create_grant(project_id=f47f695da6654f48a1df25b7f4bf15eb, user_id=81a7f8d5b07645229d40b5b27a52049c, role_id=17973d7fca9143289f3ca113a29e4166)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:22.920435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffa836-1970-4b46-a1ec-9b169638189c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:22.993602 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7be41f-2cea-4620-934b-f6e3679fb84c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:22.994311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 87/168] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:50:22 2022] PUT /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/users/81a7f8d5b07645229d40b5b27a52049c/roles/17973d7fca9143289f3ca113a29e4166 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:23.001600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffa836-1970-4b46-a1ec-9b169638189c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:23.042634 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef812c1-f243-4885-9bc9-8708736d457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:23.043351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef812c1-f243-4885-9bc9-8708736d457e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:23.043661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef812c1-f243-4885-9bc9-8708736d457e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:23.043926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef812c1-f243-4885-9bc9-8708736d457e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:23.058580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef812c1-f243-4885-9bc9-8708736d457e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4ef812c1-f243-4885-9bc9-8708736d457e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:23.064215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:23.065407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:23.066196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 82/169] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:23.073820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef812c1-f243-4885-9bc9-8708736d457e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:23.112898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:23.167124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffa836-1970-4b46-a1ec-9b169638189c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:23.190392 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:23.208163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef812c1-f243-4885-9bc9-8708736d457e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:23.210441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53a7af-5803-4c19-9556-0ae66c55ef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:23.211802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53a7af-5803-4c19-9556-0ae66c55ef47 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:23.211802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53a7af-5803-4c19-9556-0ae66c55ef47 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:23.211802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53a7af-5803-4c19-9556-0ae66c55ef47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:23.228464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53a7af-5803-4c19-9556-0ae66c55ef47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8e53a7af-5803-4c19-9556-0ae66c55ef47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:23.235924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:23.236920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:23.237985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:23.239885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 88/170] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:23.246399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53a7af-5803-4c19-9556-0ae66c55ef47 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:23.253039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18f774-9739-497e-8adc-71f89799611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:23.253850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18f774-9739-497e-8adc-71f89799611a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:23.254299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18f774-9739-497e-8adc-71f89799611a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:23.254693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18f774-9739-497e-8adc-71f89799611a None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:23.255500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18f774-9739-497e-8adc-71f89799611a None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:23.255938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18f774-9739-497e-8adc-71f89799611a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:23.257610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18f774-9739-497e-8adc-71f89799611a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:23.269142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 83/171] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:23.275553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18f774-9739-497e-8adc-71f89799611a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:23.391726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53a7af-5803-4c19-9556-0ae66c55ef47 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:23.414521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:23.453025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18f774-9739-497e-8adc-71f89799611a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:23.474918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d15e6-eb70-441c-9c4b-16f6d72b0abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:23.475576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d15e6-eb70-441c-9c4b-16f6d72b0abb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:23.475859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d15e6-eb70-441c-9c4b-16f6d72b0abb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:23.476130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d15e6-eb70-441c-9c4b-16f6d72b0abb None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:23.476820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d15e6-eb70-441c-9c4b-16f6d72b0abb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:23.477121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d15e6-eb70-441c-9c4b-16f6d72b0abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:23.478522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d15e6-eb70-441c-9c4b-16f6d72b0abb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:23.496586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 89/172] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:23.502261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d15e6-eb70-441c-9c4b-16f6d72b0abb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:23.538225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a8e1b-adf7-4420-a78d-e25936b886c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:23.538877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a8e1b-adf7-4420-a78d-e25936b886c7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:23.540523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a8e1b-adf7-4420-a78d-e25936b886c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:23.540523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a8e1b-adf7-4420-a78d-e25936b886c7 None admin] PATH_INFO: `/v3/users/a5e74655bc0d48558528bf252f846eda` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:23.540523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a8e1b-adf7-4420-a78d-e25936b886c7 None admin] RBAC: Authorizing `identity:get_user(user_id=a5e74655bc0d48558528bf252f846eda)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:23.541977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a8e1b-adf7-4420-a78d-e25936b886c7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:23.544066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 84/173] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:23 2022] GET /identity/v3/users/a5e74655bc0d48558528bf252f846eda => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:23.550586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a8e1b-adf7-4420-a78d-e25936b886c7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:23.594730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:23.668523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d15e6-eb70-441c-9c4b-16f6d72b0abb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:23.725498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb90f6-4132-4586-a22e-abc7f09b7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:23.731449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb90f6-4132-4586-a22e-abc7f09b7df2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:23.731978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb90f6-4132-4586-a22e-abc7f09b7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:23.732445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb90f6-4132-4586-a22e-abc7f09b7df2 None admin] PATH_INFO: `/v3/users/a5e74655bc0d48558528bf252f846eda` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:23.733898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb90f6-4132-4586-a22e-abc7f09b7df2 None admin] RBAC: Authorizing `identity:get_user(user_id=a5e74655bc0d48558528bf252f846eda)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:23.735695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb90f6-4132-4586-a22e-abc7f09b7df2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:23.737526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 90/174] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:23 2022] GET /identity/v3/users/a5e74655bc0d48558528bf252f846eda => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:23.744799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb90f6-4132-4586-a22e-abc7f09b7df2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:23.795700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a8e1b-adf7-4420-a78d-e25936b886c7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:23.875729 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905afa6-4cb7-43dd-a953-ccfc3f64beb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:23.876331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905afa6-4cb7-43dd-a953-ccfc3f64beb7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:23.876629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905afa6-4cb7-43dd-a953-ccfc3f64beb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:23.876897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905afa6-4cb7-43dd-a953-ccfc3f64beb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:23.925217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:23.990717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb90f6-4132-4586-a22e-abc7f09b7df2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:24.013038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:24.021825 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905afa6-4cb7-43dd-a953-ccfc3f64beb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:24.023507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905afa6-4cb7-43dd-a953-ccfc3f64beb7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:24.073381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d74e2-e60e-488d-8423-4374ee8db5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:24.074298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d74e2-e60e-488d-8423-4374ee8db5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:24.074568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d74e2-e60e-488d-8423-4374ee8db5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:24.074829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d74e2-e60e-488d-8423-4374ee8db5c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:24.214923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d74e2-e60e-488d-8423-4374ee8db5c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:24.216687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d74e2-e60e-488d-8423-4374ee8db5c3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:24.255368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 85/175] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 17:50:24.261100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905afa6-4cb7-43dd-a953-ccfc3f64beb7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:24.368153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:24.385136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 91/176] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 17:50:24.392554 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d74e2-e60e-488d-8423-4374ee8db5c3 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:24.410929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:24.486993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905afa6-4cb7-43dd-a953-ccfc3f64beb7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:24.545827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d74e2-e60e-488d-8423-4374ee8db5c3 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:24.565223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300c044-0e5e-407a-aa04-cbf985e09e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:24.565873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300c044-0e5e-407a-aa04-cbf985e09e47 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:24.566146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300c044-0e5e-407a-aa04-cbf985e09e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:24.566411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300c044-0e5e-407a-aa04-cbf985e09e47 None admin] PATH_INFO: `/v3/projects/4c7a14df803f48cfac92b8f9d9bf7c3d` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:24.567592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300c044-0e5e-407a-aa04-cbf985e09e47 None admin] RBAC: Authorizing `identity:get_project(project_id=4c7a14df803f48cfac92b8f9d9bf7c3d)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:24.569074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300c044-0e5e-407a-aa04-cbf985e09e47 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:24.571060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 86/177] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:24 2022] GET /identity/v3/projects/4c7a14df803f48cfac92b8f9d9bf7c3d => generated 312 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:24.577599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300c044-0e5e-407a-aa04-cbf985e09e47 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:24.586832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6dbb8-e8ac-47e5-8190-500be9169a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:24.587724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6dbb8-e8ac-47e5-8190-500be9169a62 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:24.588325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6dbb8-e8ac-47e5-8190-500be9169a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:24.588814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6dbb8-e8ac-47e5-8190-500be9169a62 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:24.590159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6dbb8-e8ac-47e5-8190-500be9169a62 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:24.592340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6dbb8-e8ac-47e5-8190-500be9169a62 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:24.595123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 92/178] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:24 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:24.618253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6dbb8-e8ac-47e5-8190-500be9169a62 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:24.762483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:24.768102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300c044-0e5e-407a-aa04-cbf985e09e47 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:24.785943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6dbb8-e8ac-47e5-8190-500be9169a62 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:24.803135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:24.835012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a26805-f539-4b44-a2f4-038790948e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:24.835606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a26805-f539-4b44-a2f4-038790948e7a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:24.835872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a26805-f539-4b44-a2f4-038790948e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:24.836133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a26805-f539-4b44-a2f4-038790948e7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:24.837552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a26805-f539-4b44-a2f4-038790948e7a None admin] RBAC: Adding query filter params (role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=4c7a14df803f48cfac92b8f9d9bf7c3d, user.id=a5e74655bc0d48558528bf252f846eda) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:24.840421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a26805-f539-4b44-a2f4-038790948e7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:24.844527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a26805-f539-4b44-a2f4-038790948e7a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:24.857187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 87/179] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:24 2022] GET /identity/v3/role_assignments?user.id=a5e74655bc0d48558528bf252f846eda&scope.project.id=4c7a14df803f48cfac92b8f9d9bf7c3d&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:24.860819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4336a9-746f-4671-bfaf-1d2795a7e4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:24.861424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4336a9-746f-4671-bfaf-1d2795a7e4cc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:24.861693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4336a9-746f-4671-bfaf-1d2795a7e4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:24.861956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4336a9-746f-4671-bfaf-1d2795a7e4cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:24.862687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4336a9-746f-4671-bfaf-1d2795a7e4cc None admin] RBAC: Adding query filter params (role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb, user.id=a5e74655bc0d48558528bf252f846eda) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:24.862988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4336a9-746f-4671-bfaf-1d2795a7e4cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:24.864451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4336a9-746f-4671-bfaf-1d2795a7e4cc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:24.871170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 93/180] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:24 2022] GET /identity/v3/role_assignments?user.id=a5e74655bc0d48558528bf252f846eda&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:25.435111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2bcf8-6674-4ec5-a76b-0586e0e38452 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:25.435437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2bcf8-6674-4ec5-a76b-0586e0e38452 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:25.435653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2bcf8-6674-4ec5-a76b-0586e0e38452 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:25.437274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 88/181] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:25.450102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b22de9-922c-491e-9d1c-29bf79ed18c5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:25.450380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b22de9-922c-491e-9d1c-29bf79ed18c5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:25.450648 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b22de9-922c-491e-9d1c-29bf79ed18c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:25.470927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f4b22de9-922c-491e-9d1c-29bf79ed18c5 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:25.472129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f4b22de9-922c-491e-9d1c-29bf79ed18c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:25.549901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 94/182] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:25.572898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2bcf8-6674-4ec5-a76b-0586e0e38452 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:25.710104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4127b1e-ed05-4912-8295-8c9e01d1ea8a None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:25.710274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4127b1e-ed05-4912-8295-8c9e01d1ea8a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:25.710548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4127b1e-ed05-4912-8295-8c9e01d1ea8a None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:25.711894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 95/183] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:25.716852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419ef90-a159-4396-9724-8623244d2959 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:25.717159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419ef90-a159-4396-9724-8623244d2959 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:25.717376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419ef90-a159-4396-9724-8623244d2959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:25.730745 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2bcf8-6674-4ec5-a76b-0586e0e38452 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:25.736427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f419ef90-a159-4396-9724-8623244d2959 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:25.737669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f419ef90-a159-4396-9724-8623244d2959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:25.775170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5f37d-5256-4bb2-95fc-eceb174ad488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:25.775796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5f37d-5256-4bb2-95fc-eceb174ad488 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:25.776062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5f37d-5256-4bb2-95fc-eceb174ad488 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:25.776322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5f37d-5256-4bb2-95fc-eceb174ad488 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:25.787760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5f37d-5256-4bb2-95fc-eceb174ad488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f6a5f37d-5256-4bb2-95fc-eceb174ad488 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:25.792485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:25.793419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:25.794214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 89/184] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:25.794878 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50220fb9-85c7-456e-aff9-8e58d56e09e3 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:25.795306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50220fb9-85c7-456e-aff9-8e58d56e09e3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:25.795306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50220fb9-85c7-456e-aff9-8e58d56e09e3 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:25.797439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 90/185] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:25.801346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50220fb9-85c7-456e-aff9-8e58d56e09e3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:25.828809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 96/186] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:25.833859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5bed6-74f8-461f-868c-33127cbd2e4c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:25.834139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5bed6-74f8-461f-868c-33127cbd2e4c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:25.834409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5bed6-74f8-461f-868c-33127cbd2e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:25.842994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:25.860216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:25.860525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-66b5bed6-74f8-461f-868c-33127cbd2e4c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:25.861960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-66b5bed6-74f8-461f-868c-33127cbd2e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:25.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50220fb9-85c7-456e-aff9-8e58d56e09e3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:25.930803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 97/187] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:25.931752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5bed6-74f8-461f-868c-33127cbd2e4c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:25.958234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7595d-da01-450f-a4bb-3a54e2bd8006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:25.958837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7595d-da01-450f-a4bb-3a54e2bd8006 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:25.959107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7595d-da01-450f-a4bb-3a54e2bd8006 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:25.959378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7595d-da01-450f-a4bb-3a54e2bd8006 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:25.960024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7595d-da01-450f-a4bb-3a54e2bd8006 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:25.960367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7595d-da01-450f-a4bb-3a54e2bd8006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:25.961721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7595d-da01-450f-a4bb-3a54e2bd8006 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:25.970080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 91/188] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:25.972779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c27139-9c9f-4558-8dd5-a40a97b4801f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:25.973065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c27139-9c9f-4558-8dd5-a40a97b4801f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:25.973335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c27139-9c9f-4558-8dd5-a40a97b4801f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:25.974994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 92/189] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:25.976006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c27139-9c9f-4558-8dd5-a40a97b4801f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.030224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5bed6-74f8-461f-868c-33127cbd2e4c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.058831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:26.063443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c800c-170d-4ee4-b1a9-795a45687913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.064051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c800c-170d-4ee4-b1a9-795a45687913 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.064319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c800c-170d-4ee4-b1a9-795a45687913 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.064608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c800c-170d-4ee4-b1a9-795a45687913 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.074665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c800c-170d-4ee4-b1a9-795a45687913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:26.075473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c27139-9c9f-4558-8dd5-a40a97b4801f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.077859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-498c800c-170d-4ee4-b1a9-795a45687913 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:26.080303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:26.081482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 98/190] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:26.082312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c800c-170d-4ee4-b1a9-795a45687913 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.114651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870adc84-9407-453d-9d00-806e736c55fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.115270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870adc84-9407-453d-9d00-806e736c55fc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.115537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870adc84-9407-453d-9d00-806e736c55fc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.115795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870adc84-9407-453d-9d00-806e736c55fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.126857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870adc84-9407-453d-9d00-806e736c55fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-870adc84-9407-453d-9d00-806e736c55fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:26.131489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:26.132391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:26.133492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 93/191] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:26.135053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324f7d5-64a4-4cc2-b555-12fe5ae79a61 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.135322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324f7d5-64a4-4cc2-b555-12fe5ae79a61 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.135585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324f7d5-64a4-4cc2-b555-12fe5ae79a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.153151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f324f7d5-64a4-4cc2-b555-12fe5ae79a61 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:26.154315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f324f7d5-64a4-4cc2-b555-12fe5ae79a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:26.188009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c800c-170d-4ee4-b1a9-795a45687913 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.220159 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 94/192] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:26.220985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324f7d5-64a4-4cc2-b555-12fe5ae79a61 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.236190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd9e5b-245c-4bce-b21e-b4a54f75c653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.236910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd9e5b-245c-4bce-b21e-b4a54f75c653 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.237277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd9e5b-245c-4bce-b21e-b4a54f75c653 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.237618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd9e5b-245c-4bce-b21e-b4a54f75c653 None admin] PATH_INFO: `/v3/users/c453b9ad63d5411faa3c77d19026020d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.252065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd9e5b-245c-4bce-b21e-b4a54f75c653 None admin] RBAC: Authorizing `identity:get_user(user_id=c453b9ad63d5411faa3c77d19026020d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:26.253559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd9e5b-245c-4bce-b21e-b4a54f75c653 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:26.255251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 99/193] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:26 2022] GET /identity/v3/users/c453b9ad63d5411faa3c77d19026020d => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:26.256061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd9e5b-245c-4bce-b21e-b4a54f75c653 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.307686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:26.314691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:26.321000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324f7d5-64a4-4cc2-b555-12fe5ae79a61 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.354459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871388a5-fc86-49da-8c1b-43a95bfb49df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.355053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871388a5-fc86-49da-8c1b-43a95bfb49df None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.355325 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871388a5-fc86-49da-8c1b-43a95bfb49df None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.355645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871388a5-fc86-49da-8c1b-43a95bfb49df None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.355835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd9e5b-245c-4bce-b21e-b4a54f75c653 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.356300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871388a5-fc86-49da-8c1b-43a95bfb49df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:26.356649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871388a5-fc86-49da-8c1b-43a95bfb49df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:26.358027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871388a5-fc86-49da-8c1b-43a95bfb49df None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:26.365682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 95/194] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:26.366562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871388a5-fc86-49da-8c1b-43a95bfb49df None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.393416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8d4d3-c71b-4997-ba56-d77494606025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.394008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8d4d3-c71b-4997-ba56-d77494606025 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8d4d3-c71b-4997-ba56-d77494606025 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.394538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8d4d3-c71b-4997-ba56-d77494606025 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.395193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8d4d3-c71b-4997-ba56-d77494606025 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:26.395486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8d4d3-c71b-4997-ba56-d77494606025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:26.396906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8d4d3-c71b-4997-ba56-d77494606025 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:26.404591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 100/195] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:26.405365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8d4d3-c71b-4997-ba56-d77494606025 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.466180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871388a5-fc86-49da-8c1b-43a95bfb49df None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.497723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff49f4-4ea2-4b66-874b-3f4bda20ac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.498274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff49f4-4ea2-4b66-874b-3f4bda20ac92 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.498546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff49f4-4ea2-4b66-874b-3f4bda20ac92 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.498813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff49f4-4ea2-4b66-874b-3f4bda20ac92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.500421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff49f4-4ea2-4b66-874b-3f4bda20ac92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:26.501482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8d4d3-c71b-4997-ba56-d77494606025 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.501952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff49f4-4ea2-4b66-874b-3f4bda20ac92 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:26.503510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 96/196] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:26.503510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff49f4-4ea2-4b66-874b-3f4bda20ac92 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.515490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:26.530592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:26.535076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aec1db-f37b-41e4-a394-e435e17ad6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.535715 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aec1db-f37b-41e4-a394-e435e17ad6cf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.535956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aec1db-f37b-41e4-a394-e435e17ad6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.536217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aec1db-f37b-41e4-a394-e435e17ad6cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.603920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aec1db-f37b-41e4-a394-e435e17ad6cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:26.604667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff49f4-4ea2-4b66-874b-3f4bda20ac92 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.605480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aec1db-f37b-41e4-a394-e435e17ad6cf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:26.638414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd057a0-f0fe-4fcb-954c-6eea72b05748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.639047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd057a0-f0fe-4fcb-954c-6eea72b05748 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.639318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd057a0-f0fe-4fcb-954c-6eea72b05748 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.639580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd057a0-f0fe-4fcb-954c-6eea72b05748 None admin] PATH_INFO: `/v3/users/c453b9ad63d5411faa3c77d19026020d` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.640765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd057a0-f0fe-4fcb-954c-6eea72b05748 None admin] RBAC: Authorizing `identity:get_user(user_id=c453b9ad63d5411faa3c77d19026020d)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:26.642190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd057a0-f0fe-4fcb-954c-6eea72b05748 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:26.643750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 97/197] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:26 2022] GET /identity/v3/users/c453b9ad63d5411faa3c77d19026020d => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:26.645375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd057a0-f0fe-4fcb-954c-6eea72b05748 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.709226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 101/198] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:26.709226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aec1db-f37b-41e4-a394-e435e17ad6cf None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.720123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:26.739258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:26.747255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd057a0-f0fe-4fcb-954c-6eea72b05748 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.779836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ead71-1f46-45f9-acbb-73cacc7d9976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.780383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ead71-1f46-45f9-acbb-73cacc7d9976 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.780693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ead71-1f46-45f9-acbb-73cacc7d9976 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.780956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ead71-1f46-45f9-acbb-73cacc7d9976 None admin] PATH_INFO: `/v3/users/81a7f8d5b07645229d40b5b27a52049c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.782612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ead71-1f46-45f9-acbb-73cacc7d9976 None admin] RBAC: Authorizing `identity:get_user(user_id=81a7f8d5b07645229d40b5b27a52049c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:26.784043 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ead71-1f46-45f9-acbb-73cacc7d9976 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:26.785628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 98/199] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:26 2022] GET /identity/v3/users/81a7f8d5b07645229d40b5b27a52049c => generated 289 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:26.786373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ead71-1f46-45f9-acbb-73cacc7d9976 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.808602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aec1db-f37b-41e4-a394-e435e17ad6cf None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.841875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e45f7-ef84-4ed4-84dd-ca77e2c8852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.842468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e45f7-ef84-4ed4-84dd-ca77e2c8852a None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.842717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e45f7-ef84-4ed4-84dd-ca77e2c8852a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.842977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e45f7-ef84-4ed4-84dd-ca77e2c8852a None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.843604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e45f7-ef84-4ed4-84dd-ca77e2c8852a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:26.845584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e45f7-ef84-4ed4-84dd-ca77e2c8852a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:26.882981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ead71-1f46-45f9-acbb-73cacc7d9976 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:26.915803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce89be-b964-4f02-af21-f180add9ba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:26.916397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce89be-b964-4f02-af21-f180add9ba0b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:26.916692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce89be-b964-4f02-af21-f180add9ba0b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:26.916954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce89be-b964-4f02-af21-f180add9ba0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:26.936332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 102/200] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:50:26 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:26.944664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e45f7-ef84-4ed4-84dd-ca77e2c8852a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:26.954300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:27.001236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce89be-b964-4f02-af21-f180add9ba0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:27.002830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce89be-b964-4f02-af21-f180add9ba0b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:27.013936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:27.052745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e45f7-ef84-4ed4-84dd-ca77e2c8852a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:27.093290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ca7a1-82dc-4bd3-9a88-1a9998bb74e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:27.094861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ca7a1-82dc-4bd3-9a88-1a9998bb74e7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:27.095333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ca7a1-82dc-4bd3-9a88-1a9998bb74e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:27.095598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ca7a1-82dc-4bd3-9a88-1a9998bb74e7 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:27.096760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ca7a1-82dc-4bd3-9a88-1a9998bb74e7 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:27.098207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ca7a1-82dc-4bd3-9a88-1a9998bb74e7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:27.099876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 103/201] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:26 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:27.104799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ca7a1-82dc-4bd3-9a88-1a9998bb74e7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:27.117790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 99/202] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:27.119066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce89be-b964-4f02-af21-f180add9ba0b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:27.193885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:27.226032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ca7a1-82dc-4bd3-9a88-1a9998bb74e7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:27.244797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce89be-b964-4f02-af21-f180add9ba0b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:27.269738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770c2a4-1d01-4bb0-827b-be4d307b7111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:27.270350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770c2a4-1d01-4bb0-827b-be4d307b7111 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:27.270622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770c2a4-1d01-4bb0-827b-be4d307b7111 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:27.270884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770c2a4-1d01-4bb0-827b-be4d307b7111 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:27.274513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:27.287418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50adf5b-0d23-4c28-a2f2-bb509aaeb165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:27.288018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50adf5b-0d23-4c28-a2f2-bb509aaeb165 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:27.288288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50adf5b-0d23-4c28-a2f2-bb509aaeb165 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:27.288592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50adf5b-0d23-4c28-a2f2-bb509aaeb165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:27.289316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50adf5b-0d23-4c28-a2f2-bb509aaeb165 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=00da5d0d955743658d1eb4ac7e00247f, user.id=c453b9ad63d5411faa3c77d19026020d) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:27.289615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50adf5b-0d23-4c28-a2f2-bb509aaeb165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:27.291081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50adf5b-0d23-4c28-a2f2-bb509aaeb165 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:27.299122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 100/203] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:27 2022] GET /identity/v3/role_assignments?user.id=c453b9ad63d5411faa3c77d19026020d&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:27.299122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50adf5b-0d23-4c28-a2f2-bb509aaeb165 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:27.379467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770c2a4-1d01-4bb0-827b-be4d307b7111 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:27.381315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770c2a4-1d01-4bb0-827b-be4d307b7111 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:27.487369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50adf5b-0d23-4c28-a2f2-bb509aaeb165 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:27.516541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:27.543004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 104/204] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:27.544483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5184cb-82e1-4abb-832d-150d9f46283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:27.545314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5184cb-82e1-4abb-832d-150d9f46283d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:27.545758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5184cb-82e1-4abb-832d-150d9f46283d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:27.546179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5184cb-82e1-4abb-832d-150d9f46283d None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:27.548538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5184cb-82e1-4abb-832d-150d9f46283d None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:27.549313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770c2a4-1d01-4bb0-827b-be4d307b7111 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:27.550401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5184cb-82e1-4abb-832d-150d9f46283d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:27.552234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 101/205] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:27 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:27.561581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5184cb-82e1-4abb-832d-150d9f46283d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:27.592876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:27.670004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770c2a4-1d01-4bb0-827b-be4d307b7111 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:27.710515 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082972e1-3128-439f-b484-f392c2bbef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:27.711551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082972e1-3128-439f-b484-f392c2bbef61 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:27.712029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082972e1-3128-439f-b484-f392c2bbef61 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:27.712492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082972e1-3128-439f-b484-f392c2bbef61 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:27.713873 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082972e1-3128-439f-b484-f392c2bbef61 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:27.715813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082972e1-3128-439f-b484-f392c2bbef61 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:27.718425 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 105/206] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:27 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:27.726518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082972e1-3128-439f-b484-f392c2bbef61 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:27.787023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5184cb-82e1-4abb-832d-150d9f46283d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:27.856646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936c5df-2804-45b8-af69-13a397adab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:27.857259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936c5df-2804-45b8-af69-13a397adab78 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:27.857540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936c5df-2804-45b8-af69-13a397adab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:27.857805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936c5df-2804-45b8-af69-13a397adab78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:27.858534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936c5df-2804-45b8-af69-13a397adab78 None admin] RBAC: Adding query filter params (role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=00da5d0d955743658d1eb4ac7e00247f, user.id=c453b9ad63d5411faa3c77d19026020d) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:27.858834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936c5df-2804-45b8-af69-13a397adab78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:27.860970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936c5df-2804-45b8-af69-13a397adab78 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:27.876596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 102/207] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:27 2022] GET /identity/v3/role_assignments?user.id=c453b9ad63d5411faa3c77d19026020d&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:27.920663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082972e1-3128-439f-b484-f392c2bbef61 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:27.929962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:27.979689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f420148-2386-47ee-9ae2-333df214007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:27.980691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f420148-2386-47ee-9ae2-333df214007a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:27.981184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f420148-2386-47ee-9ae2-333df214007a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:27.981653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f420148-2386-47ee-9ae2-333df214007a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:27.982604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f420148-2386-47ee-9ae2-333df214007a None admin] RBAC: Adding query filter params (role.id=17973d7fca9143289f3ca113a29e4166, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb, user.id=81a7f8d5b07645229d40b5b27a52049c) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:27.983112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f420148-2386-47ee-9ae2-333df214007a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:27.985378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f420148-2386-47ee-9ae2-333df214007a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:27.992130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 106/208] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:27 2022] GET /identity/v3/role_assignments?user.id=81a7f8d5b07645229d40b5b27a52049c&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=17973d7fca9143289f3ca113a29e4166 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:29.315895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71f692-d564-4c5d-be6d-c375cd82922a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.316922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71f692-d564-4c5d-be6d-c375cd82922a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.317397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71f692-d564-4c5d-be6d-c375cd82922a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.318952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 103/209] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:29.324153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcbaf1-759e-4f7a-b2cc-43770e36f3b8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.324968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcbaf1-759e-4f7a-b2cc-43770e36f3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.325513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcbaf1-759e-4f7a-b2cc-43770e36f3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.336870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22138-0b07-4ead-af3a-ffcd7e46a530 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.337458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22138-0b07-4ead-af3a-ffcd7e46a530 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.337950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22138-0b07-4ead-af3a-ffcd7e46a530 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.339725 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 104/210] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:29.346582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c8961-3e10-4b79-9a1b-e46d9d4c54db None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.346582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c8961-3e10-4b79-9a1b-e46d9d4c54db None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.346582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c8961-3e10-4b79-9a1b-e46d9d4c54db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.347827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8fbcbaf1-759e-4f7a-b2cc-43770e36f3b8 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:29.349247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8fbcbaf1-759e-4f7a-b2cc-43770e36f3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:29.363647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ca3c8961-3e10-4b79-9a1b-e46d9d4c54db None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:29.364863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ca3c8961-3e10-4b79-9a1b-e46d9d4c54db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:29.416757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:29.421645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 107/211] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:29.433867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 105/212] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:29.444109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcbaf1-759e-4f7a-b2cc-43770e36f3b8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:29.448547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab90d-60b2-45b4-9d5d-addad69776ca None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.448867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab90d-60b2-45b4-9d5d-addad69776ca None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.449139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab90d-60b2-45b4-9d5d-addad69776ca None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.450660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 106/213] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:29.457776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f44415-c98d-462c-9771-77f5af7fb969 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.458349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f44415-c98d-462c-9771-77f5af7fb969 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.458815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f44415-c98d-462c-9771-77f5af7fb969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.481330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-17f44415-c98d-462c-9771-77f5af7fb969 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:29.482811 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-17f44415-c98d-462c-9771-77f5af7fb969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:29.542413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:29.546280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcbaf1-759e-4f7a-b2cc-43770e36f3b8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:29.556907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 107/214] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:29.556907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f44415-c98d-462c-9771-77f5af7fb969 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:29.586323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b1adc-ee98-4dfa-96b1-9b8314294258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:29.586866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b1adc-ee98-4dfa-96b1-9b8314294258 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.587150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b1adc-ee98-4dfa-96b1-9b8314294258 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.587406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b1adc-ee98-4dfa-96b1-9b8314294258 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.598561 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b1adc-ee98-4dfa-96b1-9b8314294258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2c4b1adc-ee98-4dfa-96b1-9b8314294258 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:29.603519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:29.604660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 108/215] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:29.605742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b1adc-ee98-4dfa-96b1-9b8314294258 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:29.698989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f44415-c98d-462c-9771-77f5af7fb969 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:29.707463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b1adc-ee98-4dfa-96b1-9b8314294258 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:29.716106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:29.736177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6c4b4-de50-42ee-9644-eb67ec147136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:29.736822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6c4b4-de50-42ee-9644-eb67ec147136 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.737093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6c4b4-de50-42ee-9644-eb67ec147136 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.737354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6c4b4-de50-42ee-9644-eb67ec147136 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.738439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6c4b4-de50-42ee-9644-eb67ec147136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:29.739870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6c4b4-de50-42ee-9644-eb67ec147136 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:29.746080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 108/216] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:29.746080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6c4b4-de50-42ee-9644-eb67ec147136 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:29.747976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9dab1-8de8-4c0b-b9ef-3de7ee9dc56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:29.749407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9dab1-8de8-4c0b-b9ef-3de7ee9dc56a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.750376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9dab1-8de8-4c0b-b9ef-3de7ee9dc56a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.750649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9dab1-8de8-4c0b-b9ef-3de7ee9dc56a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.762280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d9dab1-8de8-4c0b-b9ef-3de7ee9dc56a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f3d9dab1-8de8-4c0b-b9ef-3de7ee9dc56a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:29.768013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:29.768941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:29.769721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 109/217] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:29.771326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9dab1-8de8-4c0b-b9ef-3de7ee9dc56a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:29.803636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:29.860154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6c4b4-de50-42ee-9644-eb67ec147136 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:29.889925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9dab1-8de8-4c0b-b9ef-3de7ee9dc56a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:29.901991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a9d0d-d846-45ca-b44e-76a7347352ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:29.902644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a9d0d-d846-45ca-b44e-76a7347352ef None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.902864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a9d0d-d846-45ca-b44e-76a7347352ef None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.903130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a9d0d-d846-45ca-b44e-76a7347352ef None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.903792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a9d0d-d846-45ca-b44e-76a7347352ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:29.904090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a9d0d-d846-45ca-b44e-76a7347352ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:29.905489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a9d0d-d846-45ca-b44e-76a7347352ef None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:29.913831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 109/218] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:29.914459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a9d0d-d846-45ca-b44e-76a7347352ef None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:29.930702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585eeb0-8d64-47a4-a289-f377d9d55dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:29.931301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585eeb0-8d64-47a4-a289-f377d9d55dad None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:29.931572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585eeb0-8d64-47a4-a289-f377d9d55dad None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:29.931849 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585eeb0-8d64-47a4-a289-f377d9d55dad None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:29.932467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585eeb0-8d64-47a4-a289-f377d9d55dad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:29.933931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585eeb0-8d64-47a4-a289-f377d9d55dad None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:29.962365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:30.020534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 110/219] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:50:29 2022] POST /identity/v3/users => generated 297 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:30.020534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585eeb0-8d64-47a4-a289-f377d9d55dad None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:30.024978 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a9d0d-d846-45ca-b44e-76a7347352ef None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:30.064759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab88f8-9a2e-4f73-9ccb-ba08357c10b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:30.065390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab88f8-9a2e-4f73-9ccb-ba08357c10b1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.065667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab88f8-9a2e-4f73-9ccb-ba08357c10b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.065936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab88f8-9a2e-4f73-9ccb-ba08357c10b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.066650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab88f8-9a2e-4f73-9ccb-ba08357c10b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:30.067060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab88f8-9a2e-4f73-9ccb-ba08357c10b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:30.068519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab88f8-9a2e-4f73-9ccb-ba08357c10b1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:30.078567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 110/220] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:30.079686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922c6f1-bc3b-434b-8218-81406f3c03fe None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.080224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922c6f1-bc3b-434b-8218-81406f3c03fe None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.080494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922c6f1-bc3b-434b-8218-81406f3c03fe None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.084541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 111/221] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:30.084541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922c6f1-bc3b-434b-8218-81406f3c03fe None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:30.119738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:30.148383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585eeb0-8d64-47a4-a289-f377d9d55dad None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:30.194319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ee850-ceae-4348-83c1-c25c9e433576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:30.194319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ee850-ceae-4348-83c1-c25c9e433576 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.194319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ee850-ceae-4348-83c1-c25c9e433576 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.194319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ee850-ceae-4348-83c1-c25c9e433576 None admin] PATH_INFO: `/v3/users/af3efe086e984bbe8780f9e408c9e7ec` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.210674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ee850-ceae-4348-83c1-c25c9e433576 None admin] RBAC: Authorizing `identity:get_user(user_id=af3efe086e984bbe8780f9e408c9e7ec)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:30.212745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ee850-ceae-4348-83c1-c25c9e433576 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:30.214405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 111/222] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:30 2022] GET /identity/v3/users/af3efe086e984bbe8780f9e408c9e7ec => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:30.215568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be3eaa-ff77-46ee-88f7-1f4696fa1231 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.215826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be3eaa-ff77-46ee-88f7-1f4696fa1231 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.216095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be3eaa-ff77-46ee-88f7-1f4696fa1231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.219588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922c6f1-bc3b-434b-8218-81406f3c03fe None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:30.229939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:30.236059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-09be3eaa-ff77-46ee-88f7-1f4696fa1231 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:30.237298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-09be3eaa-ff77-46ee-88f7-1f4696fa1231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:30.264019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cd441-6f83-402f-a09d-d93b54600319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:30.265075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd441-6f83-402f-a09d-d93b54600319 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.265648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd441-6f83-402f-a09d-d93b54600319 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.265926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd441-6f83-402f-a09d-d93b54600319 None admin] PATH_INFO: `/v3/users/c453b9ad63d5411faa3c77d19026020d` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.267537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cd441-6f83-402f-a09d-d93b54600319 None admin] RBAC: Authorizing `identity:get_user(user_id=c453b9ad63d5411faa3c77d19026020d)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:30.269752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cd441-6f83-402f-a09d-d93b54600319 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:30.272054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 112/223] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:30 2022] GET /identity/v3/users/c453b9ad63d5411faa3c77d19026020d => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:30.272920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cd441-6f83-402f-a09d-d93b54600319 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:30.320105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 112/224] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:30.320899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be3eaa-ff77-46ee-88f7-1f4696fa1231 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:30.395997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cd441-6f83-402f-a09d-d93b54600319 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:30.400534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:30.441028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be3eaa-ff77-46ee-88f7-1f4696fa1231 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:30.441539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225df03-4d95-4f6f-af9a-1b279a88a7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:30.442200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225df03-4d95-4f6f-af9a-1b279a88a7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.442483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225df03-4d95-4f6f-af9a-1b279a88a7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.442746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225df03-4d95-4f6f-af9a-1b279a88a7b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.478445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeba1cd-4e35-4d2b-a61c-7c1cc4299e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:30.479230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeba1cd-4e35-4d2b-a61c-7c1cc4299e76 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.479646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeba1cd-4e35-4d2b-a61c-7c1cc4299e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.480012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeba1cd-4e35-4d2b-a61c-7c1cc4299e76 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.481334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeba1cd-4e35-4d2b-a61c-7c1cc4299e76 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:30.483001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeba1cd-4e35-4d2b-a61c-7c1cc4299e76 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:30.485020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 113/225] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:30 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:30.485721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeba1cd-4e35-4d2b-a61c-7c1cc4299e76 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:30.514389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:30.548392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225df03-4d95-4f6f-af9a-1b279a88a7b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:30.550724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225df03-4d95-4f6f-af9a-1b279a88a7b2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:30.606649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeba1cd-4e35-4d2b-a61c-7c1cc4299e76 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:30.637713 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:30.643679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be8ffd-4953-425a-916f-494632a6ef81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:30.644311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be8ffd-4953-425a-916f-494632a6ef81 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.644627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be8ffd-4953-425a-916f-494632a6ef81 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.644902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be8ffd-4953-425a-916f-494632a6ef81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.656619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be8ffd-4953-425a-916f-494632a6ef81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-99be8ffd-4953-425a-916f-494632a6ef81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:30.661389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:30.662318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:30.663165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 114/226] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:30.667949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be8ffd-4953-425a-916f-494632a6ef81 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:30.695642 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 113/227] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:30.697006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225df03-4d95-4f6f-af9a-1b279a88a7b2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:30.794547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:30.819415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be8ffd-4953-425a-916f-494632a6ef81 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:30.827831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225df03-4d95-4f6f-af9a-1b279a88a7b2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:30.860089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d012d-9061-4a3a-8c4b-d11a7d42c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:30.861386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d012d-9061-4a3a-8c4b-d11a7d42c26c None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.861867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d012d-9061-4a3a-8c4b-d11a7d42c26c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.862333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d012d-9061-4a3a-8c4b-d11a7d42c26c None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f/users/c453b9ad63d5411faa3c77d19026020d/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.865070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d012d-9061-4a3a-8c4b-d11a7d42c26c None admin] RBAC: Authorizing `identity:create_grant(project_id=00da5d0d955743658d1eb4ac7e00247f, user_id=c453b9ad63d5411faa3c77d19026020d, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:30.865956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e305e-aa82-42f3-bd6e-28b609d3ba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:30.866557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e305e-aa82-42f3-bd6e-28b609d3ba69 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:30.866827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e305e-aa82-42f3-bd6e-28b609d3ba69 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:30.867093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e305e-aa82-42f3-bd6e-28b609d3ba69 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:30.867747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d012d-9061-4a3a-8c4b-d11a7d42c26c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:30.867863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e305e-aa82-42f3-bd6e-28b609d3ba69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:30.868090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e305e-aa82-42f3-bd6e-28b609d3ba69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:30.869530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e305e-aa82-42f3-bd6e-28b609d3ba69 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:30.878737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 114/228] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:30.883892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e305e-aa82-42f3-bd6e-28b609d3ba69 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:30.940276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 115/229] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:50:30 2022] PUT /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f/users/c453b9ad63d5411faa3c77d19026020d/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:30.941512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d012d-9061-4a3a-8c4b-d11a7d42c26c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:31.011069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:31.048336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e305e-aa82-42f3-bd6e-28b609d3ba69 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:31.095173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d012d-9061-4a3a-8c4b-d11a7d42c26c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:31.130009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458d016-a665-493f-999f-1439f4419991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:31.130676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458d016-a665-493f-999f-1439f4419991 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:31.130950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458d016-a665-493f-999f-1439f4419991 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:31.131218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458d016-a665-493f-999f-1439f4419991 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:31.132461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458d016-a665-493f-999f-1439f4419991 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:31.134057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458d016-a665-493f-999f-1439f4419991 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:31.135960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 115/230] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:30 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:31.141575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458d016-a665-493f-999f-1439f4419991 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:31.143501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81eff0-a1b3-4ff5-b6db-35c9e13e294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:31.144357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81eff0-a1b3-4ff5-b6db-35c9e13e294d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:31.144862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81eff0-a1b3-4ff5-b6db-35c9e13e294d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:31.145316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81eff0-a1b3-4ff5-b6db-35c9e13e294d None admin] PATH_INFO: `/v3/users/c453b9ad63d5411faa3c77d19026020d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:31.146982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81eff0-a1b3-4ff5-b6db-35c9e13e294d None admin] RBAC: Authorizing `identity:get_user(user_id=c453b9ad63d5411faa3c77d19026020d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:31.150013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81eff0-a1b3-4ff5-b6db-35c9e13e294d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:31.152074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 116/231] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:30 2022] GET /identity/v3/users/c453b9ad63d5411faa3c77d19026020d => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:31.157311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:31.158359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81eff0-a1b3-4ff5-b6db-35c9e13e294d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:31.262250 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81eff0-a1b3-4ff5-b6db-35c9e13e294d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:31.277277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458d016-a665-493f-999f-1439f4419991 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:31.281953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:31.299085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61566841-544f-4c6b-9db8-e4894a1b02c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:31.300042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61566841-544f-4c6b-9db8-e4894a1b02c7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:31.300536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61566841-544f-4c6b-9db8-e4894a1b02c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:31.300996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61566841-544f-4c6b-9db8-e4894a1b02c7 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:31.302311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61566841-544f-4c6b-9db8-e4894a1b02c7 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:31.304005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61566841-544f-4c6b-9db8-e4894a1b02c7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:31.306173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 117/232] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:31 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:31.312001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61566841-544f-4c6b-9db8-e4894a1b02c7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:31.349843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c02d3-47ed-4261-a07b-130d4f8a0879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:31.350466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c02d3-47ed-4261-a07b-130d4f8a0879 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:31.350746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c02d3-47ed-4261-a07b-130d4f8a0879 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:31.351018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c02d3-47ed-4261-a07b-130d4f8a0879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:31.351758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c02d3-47ed-4261-a07b-130d4f8a0879 None admin] RBAC: Adding query filter params (role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=00da5d0d955743658d1eb4ac7e00247f, user.id=af3efe086e984bbe8780f9e408c9e7ec) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:31.352064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c02d3-47ed-4261-a07b-130d4f8a0879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:31.353598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c02d3-47ed-4261-a07b-130d4f8a0879 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:31.367457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 116/233] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:31 2022] GET /identity/v3/role_assignments?user.id=af3efe086e984bbe8780f9e408c9e7ec&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:31.428368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61566841-544f-4c6b-9db8-e4894a1b02c7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:31.467868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634d6ee-38c6-400b-a6ab-e8ac28a955f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:31.468610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634d6ee-38c6-400b-a6ab-e8ac28a955f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:31.468884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634d6ee-38c6-400b-a6ab-e8ac28a955f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:31.469147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634d6ee-38c6-400b-a6ab-e8ac28a955f4 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f/users/c453b9ad63d5411faa3c77d19026020d/roles/b2ded61baed84402a2fa33eda9462c6f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:31.471336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634d6ee-38c6-400b-a6ab-e8ac28a955f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=00da5d0d955743658d1eb4ac7e00247f, user_id=c453b9ad63d5411faa3c77d19026020d, role_id=b2ded61baed84402a2fa33eda9462c6f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:31.472910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634d6ee-38c6-400b-a6ab-e8ac28a955f4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:31.558676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 118/234] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:50:31 2022] PUT /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f/users/c453b9ad63d5411faa3c77d19026020d/roles/b2ded61baed84402a2fa33eda9462c6f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:50:32.461128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76552a14-5efe-4a45-966c-968d5f598360 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:32.461678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76552a14-5efe-4a45-966c-968d5f598360 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:32.461678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76552a14-5efe-4a45-966c-968d5f598360 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:32.463070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 117/235] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:32.477778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc018f4-e8db-450b-98e0-9356253a3110 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:32.478251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc018f4-e8db-450b-98e0-9356253a3110 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:32.478401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc018f4-e8db-450b-98e0-9356253a3110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:32.500931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bdc018f4-e8db-450b-98e0-9356253a3110 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:32.502134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bdc018f4-e8db-450b-98e0-9356253a3110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:32.508854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99499b8-9c36-4e18-800f-83a6cc0c3949 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:32.509136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99499b8-9c36-4e18-800f-83a6cc0c3949 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:32.509401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99499b8-9c36-4e18-800f-83a6cc0c3949 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:32.510899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 118/236] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:32.515951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36223f-4339-465b-8960-461fe90a1247 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:32.516218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36223f-4339-465b-8960-461fe90a1247 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:32.516481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36223f-4339-465b-8960-461fe90a1247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:32.546024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-de36223f-4339-465b-8960-461fe90a1247 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:32.547354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-de36223f-4339-465b-8960-461fe90a1247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:32.592166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:32.639090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 119/237] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:32.668565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:32.673961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 119/238] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:32.681949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc018f4-e8db-450b-98e0-9356253a3110 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:32.713839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36223f-4339-465b-8960-461fe90a1247 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:32.835841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc018f4-e8db-450b-98e0-9356253a3110 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:32.875844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe36b33-1b28-49c9-aaf8-18e802a8ce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:32.876467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe36b33-1b28-49c9-aaf8-18e802a8ce3c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:32.876768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe36b33-1b28-49c9-aaf8-18e802a8ce3c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:32.877033 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe36b33-1b28-49c9-aaf8-18e802a8ce3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:32.878135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe36b33-1b28-49c9-aaf8-18e802a8ce3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:32.879577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe36b33-1b28-49c9-aaf8-18e802a8ce3c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:32.881322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 120/239] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:32.886718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe36b33-1b28-49c9-aaf8-18e802a8ce3c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:32.926747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36223f-4339-465b-8960-461fe90a1247 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:32.942495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:32.993842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c614a9b-83f2-43d3-9164-dfe5a2a45bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:32.994796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c614a9b-83f2-43d3-9164-dfe5a2a45bda None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:32.995278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c614a9b-83f2-43d3-9164-dfe5a2a45bda None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:32.995729 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c614a9b-83f2-43d3-9164-dfe5a2a45bda None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.014802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c614a9b-83f2-43d3-9164-dfe5a2a45bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3c614a9b-83f2-43d3-9164-dfe5a2a45bda None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:33.020399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:33.021315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:33.022104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:33.028035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 120/240] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:33.031930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe36b33-1b28-49c9-aaf8-18e802a8ce3c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:33.034438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c614a9b-83f2-43d3-9164-dfe5a2a45bda None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:33.094189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a007c4e-fdd1-478f-8000-4de88594890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:33.095167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a007c4e-fdd1-478f-8000-4de88594890c None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:33.099049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a007c4e-fdd1-478f-8000-4de88594890c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:33.099594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a007c4e-fdd1-478f-8000-4de88594890c None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.100523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a007c4e-fdd1-478f-8000-4de88594890c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:33.102212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a007c4e-fdd1-478f-8000-4de88594890c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:33.134016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:33.176652 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c614a9b-83f2-43d3-9164-dfe5a2a45bda None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:33.194190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 121/241] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:50:32 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:33.222194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ff571-2f3b-4511-b0d7-fbd6cf69df3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:33.223165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ff571-2f3b-4511-b0d7-fbd6cf69df3a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:33.223628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ff571-2f3b-4511-b0d7-fbd6cf69df3a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:33.224150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ff571-2f3b-4511-b0d7-fbd6cf69df3a None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.225089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ff571-2f3b-4511-b0d7-fbd6cf69df3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:33.225588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ff571-2f3b-4511-b0d7-fbd6cf69df3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:33.227247 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ff571-2f3b-4511-b0d7-fbd6cf69df3a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:33.238111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 121/242] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:33.245740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a007c4e-fdd1-478f-8000-4de88594890c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:33.381083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:33.384608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a007c4e-fdd1-478f-8000-4de88594890c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:33.424715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3520b6-186e-4e12-956e-f73573f3d719 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:33.425336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3520b6-186e-4e12-956e-f73573f3d719 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:33.425810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3520b6-186e-4e12-956e-f73573f3d719 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.427715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 122/243] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:33.429688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68bed0-1a91-45cb-a00b-75a1d3728670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:33.430323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68bed0-1a91-45cb-a00b-75a1d3728670 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:33.430599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68bed0-1a91-45cb-a00b-75a1d3728670 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:33.430868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68bed0-1a91-45cb-a00b-75a1d3728670 None admin] PATH_INFO: `/v3/users/22a8fdb8002640998dddc6fad5fba173` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.436630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59714bec-4d1d-4aaa-899e-a1f433bcfc66 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:33.437185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59714bec-4d1d-4aaa-899e-a1f433bcfc66 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:33.437607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59714bec-4d1d-4aaa-899e-a1f433bcfc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.448710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68bed0-1a91-45cb-a00b-75a1d3728670 None admin] RBAC: Authorizing `identity:get_user(user_id=22a8fdb8002640998dddc6fad5fba173)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:33.450242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68bed0-1a91-45cb-a00b-75a1d3728670 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:33.451915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 122/244] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:33 2022] GET /identity/v3/users/22a8fdb8002640998dddc6fad5fba173 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:33.458198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68bed0-1a91-45cb-a00b-75a1d3728670 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:33.459500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-59714bec-4d1d-4aaa-899e-a1f433bcfc66 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:33.461846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-59714bec-4d1d-4aaa-899e-a1f433bcfc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:33.559778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 123/245] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:33.589420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68bed0-1a91-45cb-a00b-75a1d3728670 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:33.589918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59714bec-4d1d-4aaa-899e-a1f433bcfc66 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:33.629480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c17be-e6c8-4ffa-ab72-bbc0bd48db2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:33.630442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c17be-e6c8-4ffa-ab72-bbc0bd48db2c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:33.630918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c17be-e6c8-4ffa-ab72-bbc0bd48db2c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:33.631385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c17be-e6c8-4ffa-ab72-bbc0bd48db2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.670954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:33.673276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:33.721585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59714bec-4d1d-4aaa-899e-a1f433bcfc66 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:33.726489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c17be-e6c8-4ffa-ab72-bbc0bd48db2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:33.727972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c17be-e6c8-4ffa-ab72-bbc0bd48db2c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:33.766243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d8bc5-deb5-4a79-abf2-2e70d474441f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:33.767297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d8bc5-deb5-4a79-abf2-2e70d474441f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:33.768545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d8bc5-deb5-4a79-abf2-2e70d474441f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:33.769041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d8bc5-deb5-4a79-abf2-2e70d474441f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.782251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d8bc5-deb5-4a79-abf2-2e70d474441f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-843d8bc5-deb5-4a79-abf2-2e70d474441f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:33.788154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:33.789116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:33.789909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:33.792863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 124/246] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:33.798668 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d8bc5-deb5-4a79-abf2-2e70d474441f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:33.866062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 123/247] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 17:50:33.874113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c17be-e6c8-4ffa-ab72-bbc0bd48db2c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:33.941632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d8bc5-deb5-4a79-abf2-2e70d474441f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:33.948157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:33.956814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:33.982065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e81f9-c962-4ef3-ad9a-f7e9afaae013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:33.982674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e81f9-c962-4ef3-ad9a-f7e9afaae013 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:33.982941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e81f9-c962-4ef3-ad9a-f7e9afaae013 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:33.983199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e81f9-c962-4ef3-ad9a-f7e9afaae013 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.983902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e81f9-c962-4ef3-ad9a-f7e9afaae013 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:33.984198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e81f9-c962-4ef3-ad9a-f7e9afaae013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:33.985630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e81f9-c962-4ef3-ad9a-f7e9afaae013 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:33.994901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 125/248] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:33.995936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d2ed2-d600-40eb-ac7f-ce1b4657b1bb None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:33.996229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d2ed2-d600-40eb-ac7f-ce1b4657b1bb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:33.996480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d2ed2-d600-40eb-ac7f-ce1b4657b1bb None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:33.997921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 126/249] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:34.002137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d2ed2-d600-40eb-ac7f-ce1b4657b1bb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:34.005917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c17be-e6c8-4ffa-ab72-bbc0bd48db2c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:34.064216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bcab9-9e42-4356-8328-8398589080a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:34.064866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bcab9-9e42-4356-8328-8398589080a8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.065132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bcab9-9e42-4356-8328-8398589080a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.065395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bcab9-9e42-4356-8328-8398589080a8 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.068908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bcab9-9e42-4356-8328-8398589080a8 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:34.070371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bcab9-9e42-4356-8328-8398589080a8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:34.075388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 124/250] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:33 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:34.076594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6d6cf-be42-46c5-a385-02b4aa0c35a0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.076870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6d6cf-be42-46c5-a385-02b4aa0c35a0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.077132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6d6cf-be42-46c5-a385-02b4aa0c35a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.102696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-47d6d6cf-be42-46c5-a385-02b4aa0c35a0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:34.102696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-47d6d6cf-be42-46c5-a385-02b4aa0c35a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:34.154958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d2ed2-d600-40eb-ac7f-ce1b4657b1bb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:34.182149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 125/251] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:34.183100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6d6cf-be42-46c5-a385-02b4aa0c35a0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:34.191714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734a142-eb68-4a33-b8bc-c85addb503ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:34.192399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a142-eb68-4a33-b8bc-c85addb503ba None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.192944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a142-eb68-4a33-b8bc-c85addb503ba None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.193222 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a142-eb68-4a33-b8bc-c85addb503ba None admin] PATH_INFO: `/v3/users/c453b9ad63d5411faa3c77d19026020d` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.194359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734a142-eb68-4a33-b8bc-c85addb503ba None admin] RBAC: Authorizing `identity:get_user(user_id=c453b9ad63d5411faa3c77d19026020d)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:34.195802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734a142-eb68-4a33-b8bc-c85addb503ba None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:34.197703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 127/252] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:34 2022] GET /identity/v3/users/c453b9ad63d5411faa3c77d19026020d => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:34.198840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce779add-8ac0-4eb9-b9ef-201cae7ad312 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.199113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce779add-8ac0-4eb9-b9ef-201cae7ad312 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.199376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce779add-8ac0-4eb9-b9ef-201cae7ad312 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.206125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 128/253] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:34.206974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce779add-8ac0-4eb9-b9ef-201cae7ad312 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:34.257706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:34.266161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:34.306500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6d6cf-be42-46c5-a385-02b4aa0c35a0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:34.325328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce779add-8ac0-4eb9-b9ef-201cae7ad312 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:34.344073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c99a96-f58c-4025-8da3-b51acd82f30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:34.344754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c99a96-f58c-4025-8da3-b51acd82f30b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.345030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c99a96-f58c-4025-8da3-b51acd82f30b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.345299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c99a96-f58c-4025-8da3-b51acd82f30b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.346029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c99a96-f58c-4025-8da3-b51acd82f30b None admin] RBAC: Adding query filter params (role.id=17973d7fca9143289f3ca113a29e4166, scope.project.id=00da5d0d955743658d1eb4ac7e00247f, user.id=22a8fdb8002640998dddc6fad5fba173) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:34.346517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c99a96-f58c-4025-8da3-b51acd82f30b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:34.348136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c99a96-f58c-4025-8da3-b51acd82f30b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:34.355096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 126/254] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:34 2022] GET /identity/v3/role_assignments?user.id=22a8fdb8002640998dddc6fad5fba173&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=17973d7fca9143289f3ca113a29e4166 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:34.355896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c99a96-f58c-4025-8da3-b51acd82f30b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:34.368644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94530cd9-f7af-42cb-b648-864b37b6aed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:34.369578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94530cd9-f7af-42cb-b648-864b37b6aed6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.370355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94530cd9-f7af-42cb-b648-864b37b6aed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.371944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94530cd9-f7af-42cb-b648-864b37b6aed6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.485358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c99a96-f58c-4025-8da3-b51acd82f30b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:34.508301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:34.524190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791464b-e099-43e4-908e-120a9bbd1b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:34.525135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791464b-e099-43e4-908e-120a9bbd1b19 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.527102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791464b-e099-43e4-908e-120a9bbd1b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.527102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791464b-e099-43e4-908e-120a9bbd1b19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.537701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791464b-e099-43e4-908e-120a9bbd1b19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8791464b-e099-43e4-908e-120a9bbd1b19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:34.542817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:34.543796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:34.544646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 127/255] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:34.545310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ed59-396e-4203-89f4-31ea790a5e9f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.545721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ed59-396e-4203-89f4-31ea790a5e9f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.545721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ed59-396e-4203-89f4-31ea790a5e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.547420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94530cd9-f7af-42cb-b648-864b37b6aed6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:34.549070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94530cd9-f7af-42cb-b648-864b37b6aed6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:34.566128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1020ed59-396e-4203-89f4-31ea790a5e9f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:34.567389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1020ed59-396e-4203-89f4-31ea790a5e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:34.636581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:34.643005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 128/256] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:34.643866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020ed59-396e-4203-89f4-31ea790a5e9f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:34.726723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 129/257] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:34.727678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94530cd9-f7af-42cb-b648-864b37b6aed6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:34.770145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020ed59-396e-4203-89f4-31ea790a5e9f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:34.780981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:34.807659 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5bbad-3a64-4e38-85ef-2b27ab1cee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:34.807659 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5bbad-3a64-4e38-85ef-2b27ab1cee7c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.807866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5bbad-3a64-4e38-85ef-2b27ab1cee7c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.808083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5bbad-3a64-4e38-85ef-2b27ab1cee7c None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.808752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5bbad-3a64-4e38-85ef-2b27ab1cee7c None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:34.809158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5bbad-3a64-4e38-85ef-2b27ab1cee7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:34.810599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5bbad-3a64-4e38-85ef-2b27ab1cee7c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:34.819696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 129/258] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:34.820744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5bbad-3a64-4e38-85ef-2b27ab1cee7c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:34.922776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5bbad-3a64-4e38-85ef-2b27ab1cee7c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:34.958280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cce58f-2a7b-4e16-b2cb-49e9d22f3eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:34.959240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cce58f-2a7b-4e16-b2cb-49e9d22f3eca None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:34.959734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cce58f-2a7b-4e16-b2cb-49e9d22f3eca None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:34.960175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cce58f-2a7b-4e16-b2cb-49e9d22f3eca None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:34.961561 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cce58f-2a7b-4e16-b2cb-49e9d22f3eca None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:34.963212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cce58f-2a7b-4e16-b2cb-49e9d22f3eca None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:34.965064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 130/259] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:34 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:34.965990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cce58f-2a7b-4e16-b2cb-49e9d22f3eca None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:34.994580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94530cd9-f7af-42cb-b648-864b37b6aed6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:35.012865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:35.073480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cce58f-2a7b-4e16-b2cb-49e9d22f3eca None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:35.094841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788473c1-1f39-45e6-afec-34648b61e349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:35.095457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788473c1-1f39-45e6-afec-34648b61e349 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:35.095731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788473c1-1f39-45e6-afec-34648b61e349 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:35.095989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788473c1-1f39-45e6-afec-34648b61e349 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:35.110244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd346622-9beb-4fee-a5cc-94bc8ee6224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:35.111169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd346622-9beb-4fee-a5cc-94bc8ee6224f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:35.111626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd346622-9beb-4fee-a5cc-94bc8ee6224f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:35.112070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd346622-9beb-4fee-a5cc-94bc8ee6224f None admin] PATH_INFO: `/v3/users/af3efe086e984bbe8780f9e408c9e7ec` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:35.113528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd346622-9beb-4fee-a5cc-94bc8ee6224f None admin] RBAC: Authorizing `identity:get_user(user_id=af3efe086e984bbe8780f9e408c9e7ec)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:35.115283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd346622-9beb-4fee-a5cc-94bc8ee6224f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:35.117250 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 131/260] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:34 2022] GET /identity/v3/users/af3efe086e984bbe8780f9e408c9e7ec => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:35.118258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd346622-9beb-4fee-a5cc-94bc8ee6224f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:35.119496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788473c1-1f39-45e6-afec-34648b61e349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-788473c1-1f39-45e6-afec-34648b61e349 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:35.126694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:35.127653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:35.128497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:35.137140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 130/261] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:35.137140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788473c1-1f39-45e6-afec-34648b61e349 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:35.160948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:35.231245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd346622-9beb-4fee-a5cc-94bc8ee6224f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:35.241192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:35.266882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ac65b-ee59-4223-b288-2cd5d0647f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:35.267781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ac65b-ee59-4223-b288-2cd5d0647f3d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:35.268306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ac65b-ee59-4223-b288-2cd5d0647f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:35.268790 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ac65b-ee59-4223-b288-2cd5d0647f3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:35.270467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ac65b-ee59-4223-b288-2cd5d0647f3d None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=00da5d0d955743658d1eb4ac7e00247f, user.id=c453b9ad63d5411faa3c77d19026020d) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:35.270979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ac65b-ee59-4223-b288-2cd5d0647f3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:35.272679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ac65b-ee59-4223-b288-2cd5d0647f3d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:35.279653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 132/262] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:35 2022] GET /identity/v3/role_assignments?user.id=c453b9ad63d5411faa3c77d19026020d&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:35.280700 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ac65b-ee59-4223-b288-2cd5d0647f3d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:35.382609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788473c1-1f39-45e6-afec-34648b61e349 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:35.395537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ac65b-ee59-4223-b288-2cd5d0647f3d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:35.434855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29224044-cbfa-4e25-8d0f-4e9a1a4d3ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:35.435826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29224044-cbfa-4e25-8d0f-4e9a1a4d3ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:35.436333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29224044-cbfa-4e25-8d0f-4e9a1a4d3ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:35.436808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29224044-cbfa-4e25-8d0f-4e9a1a4d3ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:35.437746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29224044-cbfa-4e25-8d0f-4e9a1a4d3ef0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:35.438234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29224044-cbfa-4e25-8d0f-4e9a1a4d3ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:35.439814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29224044-cbfa-4e25-8d0f-4e9a1a4d3ef0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:35.448890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 133/263] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:35.455108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29224044-cbfa-4e25-8d0f-4e9a1a4d3ef0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:35.473902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306efcce-3c5b-4d60-99f4-8f52dd372831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:35.474617 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306efcce-3c5b-4d60-99f4-8f52dd372831 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:35.474817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306efcce-3c5b-4d60-99f4-8f52dd372831 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:35.475099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306efcce-3c5b-4d60-99f4-8f52dd372831 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:35.481349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306efcce-3c5b-4d60-99f4-8f52dd372831 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:35.482860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306efcce-3c5b-4d60-99f4-8f52dd372831 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:35.485190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 131/264] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:35 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:35.493488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306efcce-3c5b-4d60-99f4-8f52dd372831 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:35.496967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:35.572370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29224044-cbfa-4e25-8d0f-4e9a1a4d3ef0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:35.611699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a6dbf-f731-4f50-bbea-20c8aeeb6b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:35.612713 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a6dbf-f731-4f50-bbea-20c8aeeb6b12 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:35.613194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a6dbf-f731-4f50-bbea-20c8aeeb6b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:35.613662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a6dbf-f731-4f50-bbea-20c8aeeb6b12 None admin] PATH_INFO: `/v3/users/c453b9ad63d5411faa3c77d19026020d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:35.615482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a6dbf-f731-4f50-bbea-20c8aeeb6b12 None admin] RBAC: Authorizing `identity:get_user(user_id=c453b9ad63d5411faa3c77d19026020d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:35.617327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a6dbf-f731-4f50-bbea-20c8aeeb6b12 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:35.625747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 134/265] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:35 2022] GET /identity/v3/users/c453b9ad63d5411faa3c77d19026020d => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:35.625747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a6dbf-f731-4f50-bbea-20c8aeeb6b12 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:35.735759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:35.739071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a6dbf-f731-4f50-bbea-20c8aeeb6b12 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:35.742473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:35.776370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1008bbf-d019-4403-95e6-161fa4cbf259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:35.777434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1008bbf-d019-4403-95e6-161fa4cbf259 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:35.777912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1008bbf-d019-4403-95e6-161fa4cbf259 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:35.778397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1008bbf-d019-4403-95e6-161fa4cbf259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:35.783999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306efcce-3c5b-4d60-99f4-8f52dd372831 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:35.841301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e615f-13ed-44d6-bda3-05e16bec5468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:35.841888 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e615f-13ed-44d6-bda3-05e16bec5468 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:35.842166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e615f-13ed-44d6-bda3-05e16bec5468 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:35.842438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e615f-13ed-44d6-bda3-05e16bec5468 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f/users/af3efe086e984bbe8780f9e408c9e7ec/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:35.845094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e615f-13ed-44d6-bda3-05e16bec5468 None admin] RBAC: Authorizing `identity:create_grant(project_id=00da5d0d955743658d1eb4ac7e00247f, user_id=af3efe086e984bbe8780f9e408c9e7ec, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:35.846434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e615f-13ed-44d6-bda3-05e16bec5468 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:35.858516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1008bbf-d019-4403-95e6-161fa4cbf259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:35.861128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1008bbf-d019-4403-95e6-161fa4cbf259 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:35.944964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 132/266] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:50:35 2022] PUT /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f/users/af3efe086e984bbe8780f9e408c9e7ec/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:35.970248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 135/267] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:35.976403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e615f-13ed-44d6-bda3-05e16bec5468 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:36.112090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e615f-13ed-44d6-bda3-05e16bec5468 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:36.154008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:36.165777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c101d4-fe3d-421b-a746-6b90e40358f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:36.172287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c101d4-fe3d-421b-a746-6b90e40358f5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.172287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c101d4-fe3d-421b-a746-6b90e40358f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.172287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c101d4-fe3d-421b-a746-6b90e40358f5 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.172287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c101d4-fe3d-421b-a746-6b90e40358f5 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:36.172287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c101d4-fe3d-421b-a746-6b90e40358f5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:36.173566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 133/268] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:35 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:36.179698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1008bbf-d019-4403-95e6-161fa4cbf259 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:36.182204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:36.224947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c57b43-e3f2-44f1-865b-83a6d8f319f3 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.225507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c57b43-e3f2-44f1-865b-83a6d8f319f3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.225957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c57b43-e3f2-44f1-865b-83a6d8f319f3 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.227624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 134/269] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:36.232926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb8aa5-37d6-4e2e-bfc3-119c1980d9e6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.233454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb8aa5-37d6-4e2e-bfc3-119c1980d9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.233957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb8aa5-37d6-4e2e-bfc3-119c1980d9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.257791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-aecb8aa5-37d6-4e2e-bfc3-119c1980d9e6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:36.258969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-aecb8aa5-37d6-4e2e-bfc3-119c1980d9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:36.305172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1008bbf-d019-4403-95e6-161fa4cbf259 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:36.342393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae216436-8851-40f2-b743-bd230efc4b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:36.343386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216436-8851-40f2-b743-bd230efc4b5f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.343884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216436-8851-40f2-b743-bd230efc4b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.344349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216436-8851-40f2-b743-bd230efc4b5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.344920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 135/270] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:36.345567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae216436-8851-40f2-b743-bd230efc4b5f None admin] RBAC: Adding query filter params (role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=00da5d0d955743658d1eb4ac7e00247f, user.id=c453b9ad63d5411faa3c77d19026020d) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:36.346098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae216436-8851-40f2-b743-bd230efc4b5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:36.346217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7f27f-4a64-4987-b97d-96554f42693f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.346479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7f27f-4a64-4987-b97d-96554f42693f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.346750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7f27f-4a64-4987-b97d-96554f42693f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.347993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 136/271] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:36.348464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae216436-8851-40f2-b743-bd230efc4b5f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:36.353969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299e31e-50c8-4380-8b56-1345a41dfe75 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299e31e-50c8-4380-8b56-1345a41dfe75 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.354524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299e31e-50c8-4380-8b56-1345a41dfe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.357573 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 136/272] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:36 2022] GET /identity/v3/role_assignments?user.id=c453b9ad63d5411faa3c77d19026020d&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:36.373980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a299e31e-50c8-4380-8b56-1345a41dfe75 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:36.375667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a299e31e-50c8-4380-8b56-1345a41dfe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:36.379962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae216436-8851-40f2-b743-bd230efc4b5f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:36.490945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:36.517945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 137/273] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:36.535249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae216436-8851-40f2-b743-bd230efc4b5f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:36.571862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26f64a-c5fa-4816-bdbe-4cd120eb3c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:36.572498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26f64a-c5fa-4816-bdbe-4cd120eb3c31 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.572797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26f64a-c5fa-4816-bdbe-4cd120eb3c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.573197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26f64a-c5fa-4816-bdbe-4cd120eb3c31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.573740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299e31e-50c8-4380-8b56-1345a41dfe75 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:36.584060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26f64a-c5fa-4816-bdbe-4cd120eb3c31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2d26f64a-c5fa-4816-bdbe-4cd120eb3c31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:36.589582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:36.590474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:36.591296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 137/274] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:36.601507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26f64a-c5fa-4816-bdbe-4cd120eb3c31 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:36.652556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:36.698448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299e31e-50c8-4380-8b56-1345a41dfe75 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:36.718081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26f64a-c5fa-4816-bdbe-4cd120eb3c31 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:36.739919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6bccb-9ad9-4d1b-b92b-0011a3584f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:36.741297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6bccb-9ad9-4d1b-b92b-0011a3584f42 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.741747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6bccb-9ad9-4d1b-b92b-0011a3584f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.742302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6bccb-9ad9-4d1b-b92b-0011a3584f42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.744138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6bccb-9ad9-4d1b-b92b-0011a3584f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:36.745307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:36.746933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6bccb-9ad9-4d1b-b92b-0011a3584f42 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:36.753821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 138/275] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:36.754410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843d33-b458-4008-a5ae-310aaf7ceeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:36.765625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843d33-b458-4008-a5ae-310aaf7ceeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.765625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843d33-b458-4008-a5ae-310aaf7ceeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.765815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6bccb-9ad9-4d1b-b92b-0011a3584f42 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:36.765889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843d33-b458-4008-a5ae-310aaf7ceeb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.765889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843d33-b458-4008-a5ae-310aaf7ceeb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:36.765889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843d33-b458-4008-a5ae-310aaf7ceeb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:36.765889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843d33-b458-4008-a5ae-310aaf7ceeb2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:36.768302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 138/276] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:36.773450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843d33-b458-4008-a5ae-310aaf7ceeb2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:36.880999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6bccb-9ad9-4d1b-b92b-0011a3584f42 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:36.898898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843d33-b458-4008-a5ae-310aaf7ceeb2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:36.924302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa7c08-d27f-4a2a-971e-fd984d20043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:36.925706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa7c08-d27f-4a2a-971e-fd984d20043f None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.926160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa7c08-d27f-4a2a-971e-fd984d20043f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.926666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa7c08-d27f-4a2a-971e-fd984d20043f None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.927675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa7c08-d27f-4a2a-971e-fd984d20043f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:36.928782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:36.930310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa7c08-d27f-4a2a-971e-fd984d20043f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:36.935904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab588aa-f6a3-425d-bed3-6eae6633ad82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:36.936653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab588aa-f6a3-425d-bed3-6eae6633ad82 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:36.942275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab588aa-f6a3-425d-bed3-6eae6633ad82 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:36.942275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab588aa-f6a3-425d-bed3-6eae6633ad82 None admin] PATH_INFO: `/v3/users/a437d213c36b41e68f36acc7ea7c100b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:36.953931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab588aa-f6a3-425d-bed3-6eae6633ad82 None admin] RBAC: Authorizing `identity:get_user(user_id=a437d213c36b41e68f36acc7ea7c100b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:36.955675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab588aa-f6a3-425d-bed3-6eae6633ad82 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:36.957578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 139/277] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:36 2022] GET /identity/v3/users/a437d213c36b41e68f36acc7ea7c100b => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:36.963365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab588aa-f6a3-425d-bed3-6eae6633ad82 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:37.012866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:37.025256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 139/278] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:50:36 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:50:37.078940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab588aa-f6a3-425d-bed3-6eae6633ad82 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:37.098945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac534b79-58f4-47ce-8062-cdc05abf3182 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:37.099664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac534b79-58f4-47ce-8062-cdc05abf3182 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:37.100233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac534b79-58f4-47ce-8062-cdc05abf3182 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:37.101885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 140/279] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:37.108035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69425334-115d-48cf-85b4-9a11e4234659 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:37.109245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69425334-115d-48cf-85b4-9a11e4234659 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:37.109738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69425334-115d-48cf-85b4-9a11e4234659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:37.120371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ac95-278d-4d9a-b7c2-3e49490d548e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:37.121100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ac95-278d-4d9a-b7c2-3e49490d548e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:37.121427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ac95-278d-4d9a-b7c2-3e49490d548e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:37.121698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ac95-278d-4d9a-b7c2-3e49490d548e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:37.122574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086ac95-278d-4d9a-b7c2-3e49490d548e None admin] RBAC: Adding query filter params (role.id=f61a519ee1a74ddeb7cb809ee43d9825, user.id=a437d213c36b41e68f36acc7ea7c100b, scope.system=all) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:37.122889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086ac95-278d-4d9a-b7c2-3e49490d548e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:37.124394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086ac95-278d-4d9a-b7c2-3e49490d548e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:37.130870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 140/280] 10.222.0.50 () {56 vars in 1308 bytes} [Mon Aug 8 17:50:36 2022] GET /identity/v3/role_assignments?user.id=a437d213c36b41e68f36acc7ea7c100b&scope.system=all&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:37.132643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-69425334-115d-48cf-85b4-9a11e4234659 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:37.134144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-69425334-115d-48cf-85b4-9a11e4234659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:37.216117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 141/281] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:37.239267 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ac95-278d-4d9a-b7c2-3e49490d548e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:37.355653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ac95-278d-4d9a-b7c2-3e49490d548e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:37.367424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:37.392249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed2db4-d893-4278-84dc-a253564efd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:37.392959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed2db4-d893-4278-84dc-a253564efd70 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:37.393228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed2db4-d893-4278-84dc-a253564efd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:37.393491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed2db4-d893-4278-84dc-a253564efd70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:37.404893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed2db4-d893-4278-84dc-a253564efd70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-eeed2db4-d893-4278-84dc-a253564efd70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:37.410349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:37.411238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:37.412051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 141/282] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:37.419518 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69425334-115d-48cf-85b4-9a11e4234659 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:37.550304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69425334-115d-48cf-85b4-9a11e4234659 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:37.574938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:37.601982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54aab9b-d1dc-4b39-a507-1d324709e112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:37.602589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aab9b-d1dc-4b39-a507-1d324709e112 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:37.602861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aab9b-d1dc-4b39-a507-1d324709e112 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:37.603124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aab9b-d1dc-4b39-a507-1d324709e112 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:37.603960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54aab9b-d1dc-4b39-a507-1d324709e112 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:37.610307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54aab9b-d1dc-4b39-a507-1d324709e112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:37.610307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54aab9b-d1dc-4b39-a507-1d324709e112 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:37.620324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 142/283] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:37.625618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed2db4-d893-4278-84dc-a253564efd70 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:37.784866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed2db4-d893-4278-84dc-a253564efd70 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:37.835630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c15ffc-7054-4e4f-a551-fe2b4861857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:37.836638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c15ffc-7054-4e4f-a551-fe2b4861857e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:37.837127 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c15ffc-7054-4e4f-a551-fe2b4861857e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:37.837581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c15ffc-7054-4e4f-a551-fe2b4861857e None admin] PATH_INFO: `/v3/users/22a8fdb8002640998dddc6fad5fba173` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:37.838930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c15ffc-7054-4e4f-a551-fe2b4861857e None admin] RBAC: Authorizing `identity:get_user(user_id=22a8fdb8002640998dddc6fad5fba173)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:37.840622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c15ffc-7054-4e4f-a551-fe2b4861857e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:37.842691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 142/284] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:37 2022] GET /identity/v3/users/22a8fdb8002640998dddc6fad5fba173 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:37.848459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54aab9b-d1dc-4b39-a507-1d324709e112 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:38.012245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54aab9b-d1dc-4b39-a507-1d324709e112 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:38.059618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bb340-29bc-4226-a05c-a0002d105b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:38.060164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bb340-29bc-4226-a05c-a0002d105b3a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:38.060434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bb340-29bc-4226-a05c-a0002d105b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:38.060730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bb340-29bc-4226-a05c-a0002d105b3a None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:38.062412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bb340-29bc-4226-a05c-a0002d105b3a None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:38.063852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bb340-29bc-4226-a05c-a0002d105b3a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:38.066484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 143/285] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:37 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:38.071237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c15ffc-7054-4e4f-a551-fe2b4861857e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:38.156294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:38.245876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c15ffc-7054-4e4f-a551-fe2b4861857e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:38.290165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e28b7-55b2-491b-b579-ddd1c092daf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:38.291153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e28b7-55b2-491b-b579-ddd1c092daf3 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:38.291655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e28b7-55b2-491b-b579-ddd1c092daf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:38.292110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e28b7-55b2-491b-b579-ddd1c092daf3 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f/users/22a8fdb8002640998dddc6fad5fba173/roles/17973d7fca9143289f3ca113a29e4166` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:38.297806 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e28b7-55b2-491b-b579-ddd1c092daf3 None admin] RBAC: Authorizing `identity:create_grant(project_id=00da5d0d955743658d1eb4ac7e00247f, user_id=22a8fdb8002640998dddc6fad5fba173, role_id=17973d7fca9143289f3ca113a29e4166)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:38.299495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e28b7-55b2-491b-b579-ddd1c092daf3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:38.396688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 143/286] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:50:38 2022] PUT /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f/users/22a8fdb8002640998dddc6fad5fba173/roles/17973d7fca9143289f3ca113a29e4166 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:38.494541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036a348-1631-4a93-aa56-929590ef5140 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:38.494862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036a348-1631-4a93-aa56-929590ef5140 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:38.495085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036a348-1631-4a93-aa56-929590ef5140 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:38.496448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 144/287] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:38.502435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad24d4-f060-4a9a-88d0-95ee0dbf34a5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:38.503022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad24d4-f060-4a9a-88d0-95ee0dbf34a5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:38.503579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad24d4-f060-4a9a-88d0-95ee0dbf34a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:38.533677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0fad24d4-f060-4a9a-88d0-95ee0dbf34a5 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:38.535294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0fad24d4-f060-4a9a-88d0-95ee0dbf34a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:38.626945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 144/288] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:38.650148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036a348-1631-4a93-aa56-929590ef5140 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:38.718087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:38.790118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036a348-1631-4a93-aa56-929590ef5140 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:38.844487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87323adb-f7f8-4c75-85ce-b2bfb7d89349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:38.845131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87323adb-f7f8-4c75-85ce-b2bfb7d89349 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:38.845403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87323adb-f7f8-4c75-85ce-b2bfb7d89349 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:38.845668 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87323adb-f7f8-4c75-85ce-b2bfb7d89349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:38.862199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87323adb-f7f8-4c75-85ce-b2bfb7d89349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-87323adb-f7f8-4c75-85ce-b2bfb7d89349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:38.867523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:38.868410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:38.869252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 145/289] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:38.877588 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad24d4-f060-4a9a-88d0-95ee0dbf34a5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:38.922452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:39.003782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad24d4-f060-4a9a-88d0-95ee0dbf34a5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:39.039002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18eac5c-8ed0-4c9f-915f-cd2055ef0fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:39.039601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18eac5c-8ed0-4c9f-915f-cd2055ef0fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:39.039875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18eac5c-8ed0-4c9f-915f-cd2055ef0fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:39.040142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18eac5c-8ed0-4c9f-915f-cd2055ef0fe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:39.040857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18eac5c-8ed0-4c9f-915f-cd2055ef0fe3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:39.041208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18eac5c-8ed0-4c9f-915f-cd2055ef0fe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:39.042639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18eac5c-8ed0-4c9f-915f-cd2055ef0fe3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:39.051618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 145/290] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:39.056861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87323adb-f7f8-4c75-85ce-b2bfb7d89349 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:39.164716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87323adb-f7f8-4c75-85ce-b2bfb7d89349 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:39.166299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:39.209045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbd5b1-ae55-4842-b4eb-fd198e261337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:39.209632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbd5b1-ae55-4842-b4eb-fd198e261337 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:39.209914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbd5b1-ae55-4842-b4eb-fd198e261337 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:39.210186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbd5b1-ae55-4842-b4eb-fd198e261337 None admin] PATH_INFO: `/v3/users/af3efe086e984bbe8780f9e408c9e7ec` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:39.216600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bbd5b1-ae55-4842-b4eb-fd198e261337 None admin] RBAC: Authorizing `identity:get_user(user_id=af3efe086e984bbe8780f9e408c9e7ec)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:39.218691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bbd5b1-ae55-4842-b4eb-fd198e261337 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:39.220303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 146/291] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:39 2022] GET /identity/v3/users/af3efe086e984bbe8780f9e408c9e7ec => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:39.226047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18eac5c-8ed0-4c9f-915f-cd2055ef0fe3 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:39.328669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:39.339469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18eac5c-8ed0-4c9f-915f-cd2055ef0fe3 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:39.374709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168a374-1366-4785-90d6-2fa5b1b81355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:39.375313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168a374-1366-4785-90d6-2fa5b1b81355 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:39.375590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168a374-1366-4785-90d6-2fa5b1b81355 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:39.375856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168a374-1366-4785-90d6-2fa5b1b81355 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:39.461760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168a374-1366-4785-90d6-2fa5b1b81355 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:39.463134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168a374-1366-4785-90d6-2fa5b1b81355 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:39.465661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3dec7-630e-435c-9a81-fa8a5dde88d4 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:39.465938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3dec7-630e-435c-9a81-fa8a5dde88d4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:39.466205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3dec7-630e-435c-9a81-fa8a5dde88d4 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:39.467890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 147/292] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:39.473555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7ed96-b1e6-450c-806f-18cad3c3adf7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:39.473833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7ed96-b1e6-450c-806f-18cad3c3adf7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:39.474107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7ed96-b1e6-450c-806f-18cad3c3adf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:39.492867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-eec7ed96-b1e6-450c-806f-18cad3c3adf7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:39.494052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eec7ed96-b1e6-450c-806f-18cad3c3adf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:39.598554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 148/293] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:39.617788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:39.625403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7ed96-b1e6-450c-806f-18cad3c3adf7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:39.635103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 146/294] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:39.640901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168a374-1366-4785-90d6-2fa5b1b81355 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:39.844204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:39.848018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7ed96-b1e6-450c-806f-18cad3c3adf7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:39.858077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168a374-1366-4785-90d6-2fa5b1b81355 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:39.896493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f575df-a772-47bb-bb36-083315ac6c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:39.897300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f575df-a772-47bb-bb36-083315ac6c17 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:39.897405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f575df-a772-47bb-bb36-083315ac6c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:39.897645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f575df-a772-47bb-bb36-083315ac6c17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:39.907922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df05eb0-446e-419c-bfcf-9dd3dfc3d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:39.909527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df05eb0-446e-419c-bfcf-9dd3dfc3d1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:39.910029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df05eb0-446e-419c-bfcf-9dd3dfc3d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:39.910499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df05eb0-446e-419c-bfcf-9dd3dfc3d1b5 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:39.911544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f575df-a772-47bb-bb36-083315ac6c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:39.912279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df05eb0-446e-419c-bfcf-9dd3dfc3d1b5 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:39.913922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df05eb0-446e-419c-bfcf-9dd3dfc3d1b5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:39.916079 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 147/295] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:39 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a5f575df-a772-47bb-bb36-083315ac6c17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:39.916601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:39.917515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:39.918330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:39.920364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d27ba8-bb6c-47fe-9405-0431db5bdeaf None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:39.920891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d27ba8-bb6c-47fe-9405-0431db5bdeaf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:39.921414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d27ba8-bb6c-47fe-9405-0431db5bdeaf None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:39.922510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 149/296] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:39.923069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 148/297] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:39.923592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f575df-a772-47bb-bb36-083315ac6c17 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:39.924266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d27ba8-bb6c-47fe-9405-0431db5bdeaf None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:39.986939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:40.029832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d27ba8-bb6c-47fe-9405-0431db5bdeaf None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:40.044371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f575df-a772-47bb-bb36-083315ac6c17 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:40.064375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e223f-2912-4d06-88e6-48c78eb3b24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:40.065000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e223f-2912-4d06-88e6-48c78eb3b24b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.065272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e223f-2912-4d06-88e6-48c78eb3b24b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.065799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e223f-2912-4d06-88e6-48c78eb3b24b None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.066191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e223f-2912-4d06-88e6-48c78eb3b24b None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:40.066488 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e223f-2912-4d06-88e6-48c78eb3b24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:40.067971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e223f-2912-4d06-88e6-48c78eb3b24b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:40.075958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 149/298] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:40.080803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea22681-26ae-4422-8198-45469fa0b5f4 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.080803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea22681-26ae-4422-8198-45469fa0b5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.080803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea22681-26ae-4422-8198-45469fa0b5f4 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.080803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 150/299] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:40.082074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0fd25-5e22-4096-b965-860bc50e190e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.082074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0fd25-5e22-4096-b965-860bc50e190e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.082337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0fd25-5e22-4096-b965-860bc50e190e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.088386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427f029-0ffd-4f13-800b-61a68ba45df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:40.089887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427f029-0ffd-4f13-800b-61a68ba45df3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.090168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427f029-0ffd-4f13-800b-61a68ba45df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.090432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427f029-0ffd-4f13-800b-61a68ba45df3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.091155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427f029-0ffd-4f13-800b-61a68ba45df3 None admin] RBAC: Adding query filter params (role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=00da5d0d955743658d1eb4ac7e00247f, user.id=af3efe086e984bbe8780f9e408c9e7ec) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:40.091508 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427f029-0ffd-4f13-800b-61a68ba45df3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:40.093001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427f029-0ffd-4f13-800b-61a68ba45df3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:40.108693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 150/300] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:39 2022] GET /identity/v3/role_assignments?user.id=af3efe086e984bbe8780f9e408c9e7ec&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:40.108693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427f029-0ffd-4f13-800b-61a68ba45df3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:40.109794 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9fe0fd25-5e22-4096-b965-860bc50e190e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:40.111252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9fe0fd25-5e22-4096-b965-860bc50e190e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:40.139115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:40.200923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 151/301] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:40.203187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0711f3-e3da-4e13-8d8f-adb048fb4f35 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.203591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0711f3-e3da-4e13-8d8f-adb048fb4f35 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.203959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0711f3-e3da-4e13-8d8f-adb048fb4f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.239280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4d0711f3-e3da-4e13-8d8f-adb048fb4f35 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:40.239478 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427f029-0ffd-4f13-800b-61a68ba45df3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:40.240862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4d0711f3-e3da-4e13-8d8f-adb048fb4f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:40.284208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5651175-1799-4a8a-b9de-babb602cc70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:40.285237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5651175-1799-4a8a-b9de-babb602cc70f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.285576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5651175-1799-4a8a-b9de-babb602cc70f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.285959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5651175-1799-4a8a-b9de-babb602cc70f None admin] PATH_INFO: `/v3/users/a437d213c36b41e68f36acc7ea7c100b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.287171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5651175-1799-4a8a-b9de-babb602cc70f None admin] RBAC: Authorizing `identity:get_user(user_id=a437d213c36b41e68f36acc7ea7c100b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:40.288751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5651175-1799-4a8a-b9de-babb602cc70f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:40.290336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 151/302] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:40 2022] GET /identity/v3/users/a437d213c36b41e68f36acc7ea7c100b => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:40.295624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5651175-1799-4a8a-b9de-babb602cc70f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:40.306272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:40.331530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 152/303] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:40.337465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0711f3-e3da-4e13-8d8f-adb048fb4f35 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:40.409849 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:40.418759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5651175-1799-4a8a-b9de-babb602cc70f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:40.454244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5d5b2-eb69-443a-a9be-d9c9a81c03d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:40.455121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d5b2-eb69-443a-a9be-d9c9a81c03d2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.455388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d5b2-eb69-443a-a9be-d9c9a81c03d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.455653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d5b2-eb69-443a-a9be-d9c9a81c03d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.457631 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0711f3-e3da-4e13-8d8f-adb048fb4f35 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:40.467751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5d5b2-eb69-443a-a9be-d9c9a81c03d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-5ec5d5b2-eb69-443a-a9be-d9c9a81c03d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:40.473606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:40.474545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:40.475362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 152/304] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:40.476683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5d5b2-eb69-443a-a9be-d9c9a81c03d2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:40.501502 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4525ee-0f7e-441a-83d6-0d05c3f6ac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:40.502151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4525ee-0f7e-441a-83d6-0d05c3f6ac60 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.502427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4525ee-0f7e-441a-83d6-0d05c3f6ac60 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.502695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4525ee-0f7e-441a-83d6-0d05c3f6ac60 None admin] PATH_INFO: `/v3/system/users/a437d213c36b41e68f36acc7ea7c100b/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.504102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4525ee-0f7e-441a-83d6-0d05c3f6ac60 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a437d213c36b41e68f36acc7ea7c100b, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:40.505520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4525ee-0f7e-441a-83d6-0d05c3f6ac60 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:40.576797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5d5b2-eb69-443a-a9be-d9c9a81c03d2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:40.594856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 153/305] 10.222.0.50 () {58 vars in 1334 bytes} [Mon Aug 8 17:50:40 2022] PUT /identity/v3/system/users/a437d213c36b41e68f36acc7ea7c100b/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:40.594856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4525ee-0f7e-441a-83d6-0d05c3f6ac60 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:40.628965 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e3e04-10e9-4352-8401-1ea11d564721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:40.629705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e3e04-10e9-4352-8401-1ea11d564721 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.629988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e3e04-10e9-4352-8401-1ea11d564721 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.630256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e3e04-10e9-4352-8401-1ea11d564721 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.631650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e3e04-10e9-4352-8401-1ea11d564721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:40.633410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e3e04-10e9-4352-8401-1ea11d564721 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:40.635571 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 153/306] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:40.641236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e3e04-10e9-4352-8401-1ea11d564721 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:40.656790 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:40.669179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:40.738403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4525ee-0f7e-441a-83d6-0d05c3f6ac60 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:40.764173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e3e04-10e9-4352-8401-1ea11d564721 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:40.783686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ceaa2-fd12-4da8-8ce8-5cfe47c7db3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:40.784384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ceaa2-fd12-4da8-8ce8-5cfe47c7db3f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.784870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ceaa2-fd12-4da8-8ce8-5cfe47c7db3f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.785138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ceaa2-fd12-4da8-8ce8-5cfe47c7db3f None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.785983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ceaa2-fd12-4da8-8ce8-5cfe47c7db3f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:40.786280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ceaa2-fd12-4da8-8ce8-5cfe47c7db3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:40.789816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ceaa2-fd12-4da8-8ce8-5cfe47c7db3f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:40.798488 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 154/307] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:40.801569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339d1d2-1ffc-4de4-84e6-de66d4cbb770 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.801569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339d1d2-1ffc-4de4-84e6-de66d4cbb770 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.801569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339d1d2-1ffc-4de4-84e6-de66d4cbb770 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.802348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 155/308] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:40.803739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339d1d2-1ffc-4de4-84e6-de66d4cbb770 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:40.812462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79b343-283d-41fa-8b8f-a61e4d048974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:40.813090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79b343-283d-41fa-8b8f-a61e4d048974 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.813610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79b343-283d-41fa-8b8f-a61e4d048974 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.813610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79b343-283d-41fa-8b8f-a61e4d048974 None admin] PATH_INFO: `/v3/groups` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.814221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79b343-283d-41fa-8b8f-a61e4d048974 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:40.815636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79b343-283d-41fa-8b8f-a61e4d048974 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:40.889666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 154/309] 10.222.0.50 () {60 vars in 1134 bytes} [Mon Aug 8 17:50:40 2022] POST /identity/v3/groups => generated 229 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:40.890795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81c248-0918-46fc-bd70-9f2dcdf9d663 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.891076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81c248-0918-46fc-bd70-9f2dcdf9d663 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.891344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81c248-0918-46fc-bd70-9f2dcdf9d663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.917121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339d1d2-1ffc-4de4-84e6-de66d4cbb770 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:40.923113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5c81c248-0918-46fc-bd70-9f2dcdf9d663 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:40.924455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5c81c248-0918-46fc-bd70-9f2dcdf9d663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:40.927461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:40.963353 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c3988-e2c2-494c-a646-325c2338ecc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:40.964153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c3988-e2c2-494c-a646-325c2338ecc0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:40.964486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c3988-e2c2-494c-a646-325c2338ecc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:40.964846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c3988-e2c2-494c-a646-325c2338ecc0 None admin] PATH_INFO: `/v3/users/7a318df8b13f4475b8d9d1a17e026a3e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:40.981408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c3988-e2c2-494c-a646-325c2338ecc0 None admin] RBAC: Authorizing `identity:get_user(user_id=7a318df8b13f4475b8d9d1a17e026a3e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:40.983104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c3988-e2c2-494c-a646-325c2338ecc0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:40.985410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 156/310] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:40 2022] GET /identity/v3/users/7a318df8b13f4475b8d9d1a17e026a3e => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:40.991320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c3988-e2c2-494c-a646-325c2338ecc0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:41.020574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 155/311] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:41.043603 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81c248-0918-46fc-bd70-9f2dcdf9d663 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:41.056281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:41.127840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c3988-e2c2-494c-a646-325c2338ecc0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:41.160903 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81c248-0918-46fc-bd70-9f2dcdf9d663 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:41.167752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5160f6-b229-4cff-848f-b4d2f2a2f3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:41.168808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5160f6-b229-4cff-848f-b4d2f2a2f3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:41.169088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5160f6-b229-4cff-848f-b4d2f2a2f3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:41.169356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5160f6-b229-4cff-848f-b4d2f2a2f3e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:41.170080 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5160f6-b229-4cff-848f-b4d2f2a2f3e5 None admin] RBAC: Adding query filter params (role.id=17973d7fca9143289f3ca113a29e4166, user.id=7a318df8b13f4475b8d9d1a17e026a3e, scope.system=all) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:41.170874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5160f6-b229-4cff-848f-b4d2f2a2f3e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:41.173035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5160f6-b229-4cff-848f-b4d2f2a2f3e5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:41.180164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 157/312] 10.222.0.50 () {56 vars in 1308 bytes} [Mon Aug 8 17:50:40 2022] GET /identity/v3/role_assignments?user.id=7a318df8b13f4475b8d9d1a17e026a3e&scope.system=all&role.id=17973d7fca9143289f3ca113a29e4166 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:41.201437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbf9e3-e534-4456-b4a4-c1182782ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:41.202099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbf9e3-e534-4456-b4a4-c1182782ff89 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:41.202917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbf9e3-e534-4456-b4a4-c1182782ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:41.203185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbf9e3-e534-4456-b4a4-c1182782ff89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:41.217488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbf9e3-e534-4456-b4a4-c1182782ff89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c7bbf9e3-e534-4456-b4a4-c1182782ff89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:41.223179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:41.224133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:41.225551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 156/313] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:41.232997 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5160f6-b229-4cff-848f-b4d2f2a2f3e5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:41.265923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:41.351154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5160f6-b229-4cff-848f-b4d2f2a2f3e5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:41.387742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6b6fb-622f-4fd3-91fe-ab01b9783324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:41.388351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6b6fb-622f-4fd3-91fe-ab01b9783324 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:41.388663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6b6fb-622f-4fd3-91fe-ab01b9783324 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:41.388930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6b6fb-622f-4fd3-91fe-ab01b9783324 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:41.389586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6b6fb-622f-4fd3-91fe-ab01b9783324 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:41.390436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6b6fb-622f-4fd3-91fe-ab01b9783324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:41.391802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6b6fb-622f-4fd3-91fe-ab01b9783324 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:41.400135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 158/314] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:41.409594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbf9e3-e534-4456-b4a4-c1182782ff89 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:41.489978 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:41.523062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbf9e3-e534-4456-b4a4-c1182782ff89 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:41.558795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43c85e-2a4b-422f-801a-63fdfbcb5e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:41.559490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43c85e-2a4b-422f-801a-63fdfbcb5e5c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:41.559782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43c85e-2a4b-422f-801a-63fdfbcb5e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:41.560052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43c85e-2a4b-422f-801a-63fdfbcb5e5c None admin] PATH_INFO: `/v3/users/22a8fdb8002640998dddc6fad5fba173` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:41.561263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43c85e-2a4b-422f-801a-63fdfbcb5e5c None admin] RBAC: Authorizing `identity:get_user(user_id=22a8fdb8002640998dddc6fad5fba173)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:41.563220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43c85e-2a4b-422f-801a-63fdfbcb5e5c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:41.565151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 157/315] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:41 2022] GET /identity/v3/users/22a8fdb8002640998dddc6fad5fba173 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:41.573608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6b6fb-622f-4fd3-91fe-ab01b9783324 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:41.669803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:41.680077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6b6fb-622f-4fd3-91fe-ab01b9783324 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:41.719781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5481dc6-a471-422a-8fed-99bdac0f6760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:41.720382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5481dc6-a471-422a-8fed-99bdac0f6760 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:41.720688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5481dc6-a471-422a-8fed-99bdac0f6760 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:41.720956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5481dc6-a471-422a-8fed-99bdac0f6760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:41.802017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5481dc6-a471-422a-8fed-99bdac0f6760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:41.803494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5481dc6-a471-422a-8fed-99bdac0f6760 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:41.898006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:41.909545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 159/316] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:41.917726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43c85e-2a4b-422f-801a-63fdfbcb5e5c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:42.027258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43c85e-2a4b-422f-801a-63fdfbcb5e5c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:42.062636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d57582-d813-4cbb-bb48-9182c8af5e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:42.063524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d57582-d813-4cbb-bb48-9182c8af5e17 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:42.064010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d57582-d813-4cbb-bb48-9182c8af5e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:42.064463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d57582-d813-4cbb-bb48-9182c8af5e17 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:42.065777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d57582-d813-4cbb-bb48-9182c8af5e17 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:42.067512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d57582-d813-4cbb-bb48-9182c8af5e17 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:42.069741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 158/317] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:41 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:42.077177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5481dc6-a471-422a-8fed-99bdac0f6760 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:42.192114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5481dc6-a471-422a-8fed-99bdac0f6760 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:42.229562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade67dc-d9f7-4a7b-a066-5e5a405e7c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:42.230161 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade67dc-d9f7-4a7b-a066-5e5a405e7c13 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:42.230493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade67dc-d9f7-4a7b-a066-5e5a405e7c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:42.230763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade67dc-d9f7-4a7b-a066-5e5a405e7c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:42.231519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade67dc-d9f7-4a7b-a066-5e5a405e7c13 None admin] RBAC: Adding query filter params (role.id=17973d7fca9143289f3ca113a29e4166, scope.project.id=00da5d0d955743658d1eb4ac7e00247f, user.id=22a8fdb8002640998dddc6fad5fba173) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:42.231836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade67dc-d9f7-4a7b-a066-5e5a405e7c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:42.233294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade67dc-d9f7-4a7b-a066-5e5a405e7c13 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:42.239401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 160/318] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:50:42 2022] GET /identity/v3/role_assignments?user.id=22a8fdb8002640998dddc6fad5fba173&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=17973d7fca9143289f3ca113a29e4166 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:42.697280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df43fae-e94e-449d-a8d0-b98ba88bc1f2 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:42.697653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df43fae-e94e-449d-a8d0-b98ba88bc1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:42.697829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df43fae-e94e-449d-a8d0-b98ba88bc1f2 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:42.699307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 159/319] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:42.704230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b151e1-056e-4a6b-b55f-616ab3c688b0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:42.704528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b151e1-056e-4a6b-b55f-616ab3c688b0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:42.705067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b151e1-056e-4a6b-b55f-616ab3c688b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:42.722924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-28b151e1-056e-4a6b-b55f-616ab3c688b0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:42.724086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-28b151e1-056e-4a6b-b55f-616ab3c688b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:42.774877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:42.793968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 161/320] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:42.817918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df43fae-e94e-449d-a8d0-b98ba88bc1f2 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:42.827477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:42.919532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df43fae-e94e-449d-a8d0-b98ba88bc1f2 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:42.952774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e67791-a16b-47c0-9312-08bccd0a0b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:42.953313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e67791-a16b-47c0-9312-08bccd0a0b69 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:42.953582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e67791-a16b-47c0-9312-08bccd0a0b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:42.953842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e67791-a16b-47c0-9312-08bccd0a0b69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:42.963950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e67791-a16b-47c0-9312-08bccd0a0b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-b4e67791-a16b-47c0-9312-08bccd0a0b69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:42.969572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:42.970544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:42.971361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:42.972639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 160/321] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:42.976625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b151e1-056e-4a6b-b55f-616ab3c688b0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:43.077970 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b151e1-056e-4a6b-b55f-616ab3c688b0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:43.095501 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c513a-386f-4473-99fb-44e3d949e711 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.095979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c513a-386f-4473-99fb-44e3d949e711 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.096089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c513a-386f-4473-99fb-44e3d949e711 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.097509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 161/322] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:43.102148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5d174-8506-41fc-a546-d34bd2b72b63 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.102421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5d174-8506-41fc-a546-d34bd2b72b63 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.102688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5d174-8506-41fc-a546-d34bd2b72b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.114528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bcb76-312d-432c-b12b-bab790b63886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:43.115187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bcb76-312d-432c-b12b-bab790b63886 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.115461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bcb76-312d-432c-b12b-bab790b63886 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.115740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bcb76-312d-432c-b12b-bab790b63886 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.116866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bcb76-312d-432c-b12b-bab790b63886 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:43.117166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bcb76-312d-432c-b12b-bab790b63886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:43.118543 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bcb76-312d-432c-b12b-bab790b63886 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:43.121324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-54e5d174-8506-41fc-a546-d34bd2b72b63 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:43.122516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-54e5d174-8506-41fc-a546-d34bd2b72b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:43.126853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 162/323] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:43.132241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bcb76-312d-432c-b12b-bab790b63886 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:43.178290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:43.186172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:43.200800 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 162/324] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:43.216730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5d174-8506-41fc-a546-d34bd2b72b63 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:43.243442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bcb76-312d-432c-b12b-bab790b63886 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:43.276631 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ba6af-7cee-402f-9cc6-ba0e30646f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:43.277241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ba6af-7cee-402f-9cc6-ba0e30646f7b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.277511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ba6af-7cee-402f-9cc6-ba0e30646f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.277772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ba6af-7cee-402f-9cc6-ba0e30646f7b None admin] PATH_INFO: `/v3/users/a437d213c36b41e68f36acc7ea7c100b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.278877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ba6af-7cee-402f-9cc6-ba0e30646f7b None admin] RBAC: Authorizing `identity:get_user(user_id=a437d213c36b41e68f36acc7ea7c100b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:43.280879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ba6af-7cee-402f-9cc6-ba0e30646f7b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:43.282428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 163/325] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:43 2022] GET /identity/v3/users/a437d213c36b41e68f36acc7ea7c100b => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:43.287485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da4def-e965-496b-b60d-e2ef14023aac None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.287485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da4def-e965-496b-b60d-e2ef14023aac None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.287485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da4def-e965-496b-b60d-e2ef14023aac None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.287485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 164/326] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:43.289443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da4def-e965-496b-b60d-e2ef14023aac None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:43.315472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5d174-8506-41fc-a546-d34bd2b72b63 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:43.348968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b75f4-0dbb-418c-a3f1-9f2b69d9d5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:43.349583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b75f4-0dbb-418c-a3f1-9f2b69d9d5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.349860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b75f4-0dbb-418c-a3f1-9f2b69d9d5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.350133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b75f4-0dbb-418c-a3f1-9f2b69d9d5f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.351226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b75f4-0dbb-418c-a3f1-9f2b69d9d5f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:43.352703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b75f4-0dbb-418c-a3f1-9f2b69d9d5f7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:43.354600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 163/327] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:50:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:43.355825 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e84da-ada7-4d1f-a0cd-0463d12e6865 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.356106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e84da-ada7-4d1f-a0cd-0463d12e6865 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.356375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e84da-ada7-4d1f-a0cd-0463d12e6865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.373483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ec8e84da-ada7-4d1f-a0cd-0463d12e6865 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:43.374692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ec8e84da-ada7-4d1f-a0cd-0463d12e6865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:43.389063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da4def-e965-496b-b60d-e2ef14023aac None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:43.403188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:43.423304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc57a04-1708-43b9-b014-247b26f61dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:43.423894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc57a04-1708-43b9-b014-247b26f61dba None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.424182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc57a04-1708-43b9-b014-247b26f61dba None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.424493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc57a04-1708-43b9-b014-247b26f61dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.425236 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc57a04-1708-43b9-b014-247b26f61dba None admin] RBAC: Adding query filter params (role.id=f61a519ee1a74ddeb7cb809ee43d9825, user.id=a437d213c36b41e68f36acc7ea7c100b, scope.system=all) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:43.425539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc57a04-1708-43b9-b014-247b26f61dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:43.426982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc57a04-1708-43b9-b014-247b26f61dba None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:43.431795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:43.439165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 165/328] 10.222.0.50 () {56 vars in 1308 bytes} [Mon Aug 8 17:50:43 2022] GET /identity/v3/role_assignments?user.id=a437d213c36b41e68f36acc7ea7c100b&scope.system=all&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 510 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:43.439499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 164/329] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:43.439936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc57a04-1708-43b9-b014-247b26f61dba None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:43.475876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e84da-ada7-4d1f-a0cd-0463d12e6865 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:43.558689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc57a04-1708-43b9-b014-247b26f61dba None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:43.588544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e84da-ada7-4d1f-a0cd-0463d12e6865 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:43.597965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f2e2b-3092-4e06-81fb-3b93de1e4177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:43.598591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f2e2b-3092-4e06-81fb-3b93de1e4177 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.598860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f2e2b-3092-4e06-81fb-3b93de1e4177 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.599598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f2e2b-3092-4e06-81fb-3b93de1e4177 None admin] PATH_INFO: `/v3/groups` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.599729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f2e2b-3092-4e06-81fb-3b93de1e4177 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:43.601220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f2e2b-3092-4e06-81fb-3b93de1e4177 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:43.629009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bfb32-7d7a-4a61-8669-10779bbc2afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:43.629640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bfb32-7d7a-4a61-8669-10779bbc2afc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.629911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bfb32-7d7a-4a61-8669-10779bbc2afc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.630175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bfb32-7d7a-4a61-8669-10779bbc2afc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.642935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bfb32-7d7a-4a61-8669-10779bbc2afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f97bfb32-7d7a-4a61-8669-10779bbc2afc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:43.647759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:43.648840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:43.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 165/330] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:43.654762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bfb32-7d7a-4a61-8669-10779bbc2afc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:43.695955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 166/331] 10.222.0.50 () {60 vars in 1134 bytes} [Mon Aug 8 17:50:43 2022] POST /identity/v3/groups => generated 226 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:50:43.710833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:43.757899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bfb32-7d7a-4a61-8669-10779bbc2afc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:43.794200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5f27-616b-45fc-8240-0520079cd2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:43.794813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5f27-616b-45fc-8240-0520079cd2da None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.795088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5f27-616b-45fc-8240-0520079cd2da None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.795354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5f27-616b-45fc-8240-0520079cd2da None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.796593 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba5f27-616b-45fc-8240-0520079cd2da None admin] RBAC: Adding query filter params (name=reader) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:43.796893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba5f27-616b-45fc-8240-0520079cd2da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:43.799548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba5f27-616b-45fc-8240-0520079cd2da None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:43.809307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 166/332] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:43.814971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f2e2b-3092-4e06-81fb-3b93de1e4177 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:43.848154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:43.918590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f2e2b-3092-4e06-81fb-3b93de1e4177 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:43.955502 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab5fbc-1119-4e4b-a628-f807fe047ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:43.956120 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5fbc-1119-4e4b-a628-f807fe047ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:43.956389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5fbc-1119-4e4b-a628-f807fe047ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:43.956687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5fbc-1119-4e4b-a628-f807fe047ee7 None admin] PATH_INFO: `/v3/users/7a318df8b13f4475b8d9d1a17e026a3e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:43.957806 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab5fbc-1119-4e4b-a628-f807fe047ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=7a318df8b13f4475b8d9d1a17e026a3e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:43.959245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab5fbc-1119-4e4b-a628-f807fe047ee7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:43.960995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 167/333] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:43 2022] GET /identity/v3/users/7a318df8b13f4475b8d9d1a17e026a3e => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:43.966402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5f27-616b-45fc-8240-0520079cd2da None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:44.081283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5f27-616b-45fc-8240-0520079cd2da None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:44.098317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:44.118125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e23858-309a-454c-b542-86c9bc00e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:44.119091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e23858-309a-454c-b542-86c9bc00e228 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:44.119514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e23858-309a-454c-b542-86c9bc00e228 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:44.119929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e23858-309a-454c-b542-86c9bc00e228 None admin] PATH_INFO: `/v3/system/users/7a318df8b13f4475b8d9d1a17e026a3e/roles/17973d7fca9143289f3ca113a29e4166` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:44.121660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e23858-309a-454c-b542-86c9bc00e228 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7a318df8b13f4475b8d9d1a17e026a3e, role_id=17973d7fca9143289f3ca113a29e4166)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:44.123324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e23858-309a-454c-b542-86c9bc00e228 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:44.166035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 167/334] 10.222.0.50 () {58 vars in 1334 bytes} [Mon Aug 8 17:50:43 2022] PUT /identity/v3/system/users/7a318df8b13f4475b8d9d1a17e026a3e/roles/17973d7fca9143289f3ca113a29e4166 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:46.008185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c392ca-b2c6-4c9f-a048-e3177e15e4e1 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:46.008735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c392ca-b2c6-4c9f-a048-e3177e15e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:46.008735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c392ca-b2c6-4c9f-a048-e3177e15e4e1 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:46.010595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 168/335] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:46.016116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcd955-ad93-42fd-9b72-6ac082e9909f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:46.016948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcd955-ad93-42fd-9b72-6ac082e9909f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:46.017433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcd955-ad93-42fd-9b72-6ac082e9909f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:46.041473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-42dcd955-ad93-42fd-9b72-6ac082e9909f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:46.043012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-42dcd955-ad93-42fd-9b72-6ac082e9909f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:46.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 168/336] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:46.158152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c392ca-b2c6-4c9f-a048-e3177e15e4e1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:46.279373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:46.286086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c392ca-b2c6-4c9f-a048-e3177e15e4e1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:46.321372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae07a8-db0d-4a22-a67b-3edac877f8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:46.322376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae07a8-db0d-4a22-a67b-3edac877f8ec None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:46.322848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae07a8-db0d-4a22-a67b-3edac877f8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:46.323328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae07a8-db0d-4a22-a67b-3edac877f8ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:46.339630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae07a8-db0d-4a22-a67b-3edac877f8ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-78ae07a8-db0d-4a22-a67b-3edac877f8ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:46.345195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:46.346091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:46.346897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:46.349157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 169/337] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:46.355158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcd955-ad93-42fd-9b72-6ac082e9909f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:46.466526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcd955-ad93-42fd-9b72-6ac082e9909f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:46.480072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:46.516231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8ea01-eb3e-4342-8146-3f9421cc3040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:46.516864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8ea01-eb3e-4342-8146-3f9421cc3040 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:46.517137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8ea01-eb3e-4342-8146-3f9421cc3040 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:46.517401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8ea01-eb3e-4342-8146-3f9421cc3040 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:46.518076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8ea01-eb3e-4342-8146-3f9421cc3040 None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:46.518371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8ea01-eb3e-4342-8146-3f9421cc3040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:46.519771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8ea01-eb3e-4342-8146-3f9421cc3040 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:46.532998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 169/338] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:46.538641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae07a8-db0d-4a22-a67b-3edac877f8ec None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:46.621977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d6bd-70a7-4d5a-aad6-3372c85e8f3a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:46.622294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d6bd-70a7-4d5a-aad6-3372c85e8f3a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:46.622514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d6bd-70a7-4d5a-aad6-3372c85e8f3a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:46.624033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 170/339] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:46.631393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8801593-a092-4202-b3d0-d5bace472c3b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:46.631393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8801593-a092-4202-b3d0-d5bace472c3b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:46.631393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8801593-a092-4202-b3d0-d5bace472c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:46.660617 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a8801593-a092-4202-b3d0-d5bace472c3b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:46.662140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a8801593-a092-4202-b3d0-d5bace472c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:46.703979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae07a8-db0d-4a22-a67b-3edac877f8ec None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:46.771962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e271d9-1ac8-4b73-b4cb-e9f4954ec4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:46.772519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e271d9-1ac8-4b73-b4cb-e9f4954ec4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:46.773046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e271d9-1ac8-4b73-b4cb-e9f4954ec4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:46.773319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e271d9-1ac8-4b73-b4cb-e9f4954ec4d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:46.790202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 171/340] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:46.833798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:46.836192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8801593-a092-4202-b3d0-d5bace472c3b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:46.900115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e271d9-1ac8-4b73-b4cb-e9f4954ec4d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:46.901701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e271d9-1ac8-4b73-b4cb-e9f4954ec4d5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:47.027581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8801593-a092-4202-b3d0-d5bace472c3b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:47.030829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:47.094643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c7fef-9142-41d6-b79b-7dab0ff7b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:47.095345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c7fef-9142-41d6-b79b-7dab0ff7b558 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.095654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c7fef-9142-41d6-b79b-7dab0ff7b558 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.095920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c7fef-9142-41d6-b79b-7dab0ff7b558 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.118290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c7fef-9142-41d6-b79b-7dab0ff7b558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f01c7fef-9142-41d6-b79b-7dab0ff7b558 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:47.123101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:47.132407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:47.133280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 172/341] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:47.133951 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 170/342] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:47.134341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bce722-028b-45ea-8541-be141f90e194 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.134341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bce722-028b-45ea-8541-be141f90e194 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.134341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bce722-028b-45ea-8541-be141f90e194 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.134341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 173/343] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:47.134564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e271d9-1ac8-4b73-b4cb-e9f4954ec4d5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:47.134641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bce722-028b-45ea-8541-be141f90e194 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:47.216488 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:47.268374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e271d9-1ac8-4b73-b4cb-e9f4954ec4d5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:47.300675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bce722-028b-45ea-8541-be141f90e194 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:47.306073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df78e6-31c0-47fe-902c-a75a054c6680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:47.306879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df78e6-31c0-47fe-902c-a75a054c6680 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.307299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df78e6-31c0-47fe-902c-a75a054c6680 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.307708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df78e6-31c0-47fe-902c-a75a054c6680 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.308536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df78e6-31c0-47fe-902c-a75a054c6680 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:47.308982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df78e6-31c0-47fe-902c-a75a054c6680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:47.310532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df78e6-31c0-47fe-902c-a75a054c6680 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:47.321062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 171/344] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:47.322603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639deeb-53f1-44fa-97df-872daff3fc55 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.323033 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639deeb-53f1-44fa-97df-872daff3fc55 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.323441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639deeb-53f1-44fa-97df-872daff3fc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.342557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9639deeb-53f1-44fa-97df-872daff3fc55 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:47.344524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9639deeb-53f1-44fa-97df-872daff3fc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:47.352079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b21a47-5a51-4cd2-81e9-2ca87d36c0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:47.352799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b21a47-5a51-4cd2-81e9-2ca87d36c0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.353074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b21a47-5a51-4cd2-81e9-2ca87d36c0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.353340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b21a47-5a51-4cd2-81e9-2ca87d36c0e7 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.354730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b21a47-5a51-4cd2-81e9-2ca87d36c0e7 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:47.356390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b21a47-5a51-4cd2-81e9-2ca87d36c0e7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:47.359813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 174/345] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:47 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:47.362241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde7d17-ccb8-4642-af4d-451230b25fa0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.362563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde7d17-ccb8-4642-af4d-451230b25fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.362925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde7d17-ccb8-4642-af4d-451230b25fa0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.365209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 175/346] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:47.366169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde7d17-ccb8-4642-af4d-451230b25fa0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:47.410090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:47.431956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 172/347] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:47.433209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639deeb-53f1-44fa-97df-872daff3fc55 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:47.500772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:47.513830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde7d17-ccb8-4642-af4d-451230b25fa0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:47.552752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89111b-1510-4d74-b4f5-0baaac631e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:47.553441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89111b-1510-4d74-b4f5-0baaac631e5c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.553563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89111b-1510-4d74-b4f5-0baaac631e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.554150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89111b-1510-4d74-b4f5-0baaac631e5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.564565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639deeb-53f1-44fa-97df-872daff3fc55 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:47.609174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10414f1-7ee1-4202-8564-d9949925f0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:47.609890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10414f1-7ee1-4202-8564-d9949925f0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.610163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10414f1-7ee1-4202-8564-d9949925f0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.610426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10414f1-7ee1-4202-8564-d9949925f0e4 None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.617991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10414f1-7ee1-4202-8564-d9949925f0e4 None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:47.619469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10414f1-7ee1-4202-8564-d9949925f0e4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:47.621120 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 173/348] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:47 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:47.622510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc496f4-525b-41e9-b99d-669106053fd7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.622671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc496f4-525b-41e9-b99d-669106053fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.622939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc496f4-525b-41e9-b99d-669106053fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.641408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89111b-1510-4d74-b4f5-0baaac631e5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:47.642846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89111b-1510-4d74-b4f5-0baaac631e5c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:47.645322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bbc496f4-525b-41e9-b99d-669106053fd7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:47.646518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bbc496f4-525b-41e9-b99d-669106053fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:47.682604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:47.744397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 174/349] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:47.745592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc496f4-525b-41e9-b99d-669106053fd7 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:47.785127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 176/350] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:47.786506 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd159270-93d2-4f50-a236-f5872a8437aa None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.786784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd159270-93d2-4f50-a236-f5872a8437aa None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.787055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd159270-93d2-4f50-a236-f5872a8437aa None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.790512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 177/351] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:47.790512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd159270-93d2-4f50-a236-f5872a8437aa None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:47.825957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:47.900938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc496f4-525b-41e9-b99d-669106053fd7 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:47.938853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd159270-93d2-4f50-a236-f5872a8437aa None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:47.946544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306cbf4-5dbf-4b6d-9fd1-602e324a73cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:47.947246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306cbf4-5dbf-4b6d-9fd1-602e324a73cc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.947550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306cbf4-5dbf-4b6d-9fd1-602e324a73cc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.947819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306cbf4-5dbf-4b6d-9fd1-602e324a73cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.962116 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306cbf4-5dbf-4b6d-9fd1-602e324a73cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-a306cbf4-5dbf-4b6d-9fd1-602e324a73cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:47.968292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:47.969283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 17:50:47.970097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:47.971971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 175/352] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:47.975882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306cbf4-5dbf-4b6d-9fd1-602e324a73cc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:47.980068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:47.988906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c1066-faf4-4f01-a5a4-800dd3961433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:47.989820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1066-faf4-4f01-a5a4-800dd3961433 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:47.990101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1066-faf4-4f01-a5a4-800dd3961433 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:47.994158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1066-faf4-4f01-a5a4-800dd3961433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:47.994158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1066-faf4-4f01-a5a4-800dd3961433 None admin] RBAC: Adding query filter params (group.id=130ab88bb892478198b0e3e61da62fcc, role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:47.994158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1066-faf4-4f01-a5a4-800dd3961433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:47.994158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1066-faf4-4f01-a5a4-800dd3961433 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:47.999871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 178/353] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:47 2022] GET /identity/v3/role_assignments?group.id=130ab88bb892478198b0e3e61da62fcc&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:48.000901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0caa7-37eb-479a-a060-8830724c7d78 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.001220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0caa7-37eb-479a-a060-8830724c7d78 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.002818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0caa7-37eb-479a-a060-8830724c7d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.027530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6dd0caa7-37eb-479a-a060-8830724c7d78 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:48.028936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6dd0caa7-37eb-479a-a060-8830724c7d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:48.112325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306cbf4-5dbf-4b6d-9fd1-602e324a73cc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:48.121510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:48.125975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 179/354] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:48.127203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0caa7-37eb-479a-a060-8830724c7d78 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:48.187129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ef059-c583-47db-ba13-a2085beb5f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:48.187779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ef059-c583-47db-ba13-a2085beb5f17 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.188052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ef059-c583-47db-ba13-a2085beb5f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.188319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ef059-c583-47db-ba13-a2085beb5f17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.208931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ef059-c583-47db-ba13-a2085beb5f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9e3ef059-c583-47db-ba13-a2085beb5f17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:48.215516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:48.216397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:48.217218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:48.220211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 176/355] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:48.221343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ef059-c583-47db-ba13-a2085beb5f17 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:48.274080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0caa7-37eb-479a-a060-8830724c7d78 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:48.305881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:48.313264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2b6aa-ab61-48f3-ba1b-8341acff3233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:48.313900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b6aa-ab61-48f3-ba1b-8341acff3233 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.314204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b6aa-ab61-48f3-ba1b-8341acff3233 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.314542 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b6aa-ab61-48f3-ba1b-8341acff3233 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.315880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2b6aa-ab61-48f3-ba1b-8341acff3233 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:48.317362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2b6aa-ab61-48f3-ba1b-8341acff3233 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:48.319309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 180/356] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:48 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:48.324238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2b6aa-ab61-48f3-ba1b-8341acff3233 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:48.360340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ef059-c583-47db-ba13-a2085beb5f17 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:48.409756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10111540-0f3f-43fc-bef1-c977ea155b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:48.410288 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10111540-0f3f-43fc-bef1-c977ea155b81 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.410617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10111540-0f3f-43fc-bef1-c977ea155b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.410846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10111540-0f3f-43fc-bef1-c977ea155b81 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.411522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10111540-0f3f-43fc-bef1-c977ea155b81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:48.411822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10111540-0f3f-43fc-bef1-c977ea155b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:48.413291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10111540-0f3f-43fc-bef1-c977ea155b81 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:48.422886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 177/357] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:48.423538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10111540-0f3f-43fc-bef1-c977ea155b81 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:48.458656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2b6aa-ab61-48f3-ba1b-8341acff3233 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:48.463574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:48.503409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6463fdb-c40e-4c0a-b1aa-147e15c8b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:48.504042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6463fdb-c40e-4c0a-b1aa-147e15c8b941 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.504313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6463fdb-c40e-4c0a-b1aa-147e15c8b941 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.504608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6463fdb-c40e-4c0a-b1aa-147e15c8b941 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.516328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6463fdb-c40e-4c0a-b1aa-147e15c8b941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a6463fdb-c40e-4c0a-b1aa-147e15c8b941 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:48.522353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:48.523259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:48.524150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 181/358] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:48.528715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b859cc-886b-4020-8aef-cf6b64f4da3e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.529021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b859cc-886b-4020-8aef-cf6b64f4da3e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.529291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b859cc-886b-4020-8aef-cf6b64f4da3e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.530921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 182/359] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:48.531517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b859cc-886b-4020-8aef-cf6b64f4da3e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:48.570812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10111540-0f3f-43fc-bef1-c977ea155b81 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:48.611275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a910c59-5ed9-402a-9d0c-185230d82fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:48.611886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a910c59-5ed9-402a-9d0c-185230d82fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.612165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a910c59-5ed9-402a-9d0c-185230d82fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.612430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a910c59-5ed9-402a-9d0c-185230d82fb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.613151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:48.613588 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a910c59-5ed9-402a-9d0c-185230d82fb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:48.613891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a910c59-5ed9-402a-9d0c-185230d82fb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:48.615278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a910c59-5ed9-402a-9d0c-185230d82fb6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:48.625765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 178/360] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:48.626994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a910c59-5ed9-402a-9d0c-185230d82fb6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:48.693914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b859cc-886b-4020-8aef-cf6b64f4da3e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:48.736318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626b2bf-b25a-47b6-80b3-46b7190414fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:48.736996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626b2bf-b25a-47b6-80b3-46b7190414fa None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.737269 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626b2bf-b25a-47b6-80b3-46b7190414fa None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.737534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626b2bf-b25a-47b6-80b3-46b7190414fa None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.738646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626b2bf-b25a-47b6-80b3-46b7190414fa None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:48.740850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626b2bf-b25a-47b6-80b3-46b7190414fa None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:48.741678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 183/361] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:48 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:48.742426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626b2bf-b25a-47b6-80b3-46b7190414fa None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:48.747477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:48.758876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a910c59-5ed9-402a-9d0c-185230d82fb6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:48.801709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7813497-b323-4d46-b6b6-04ef29baeb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:48.802349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7813497-b323-4d46-b6b6-04ef29baeb51 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.802626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7813497-b323-4d46-b6b6-04ef29baeb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.803499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7813497-b323-4d46-b6b6-04ef29baeb51 None admin] PATH_INFO: `/v3/users/7a318df8b13f4475b8d9d1a17e026a3e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.806497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7813497-b323-4d46-b6b6-04ef29baeb51 None admin] RBAC: Authorizing `identity:get_user(user_id=7a318df8b13f4475b8d9d1a17e026a3e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:48.808171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7813497-b323-4d46-b6b6-04ef29baeb51 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:48.809754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 179/362] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:50:48 2022] GET /identity/v3/users/7a318df8b13f4475b8d9d1a17e026a3e => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:48.812595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693310c-0ffa-46fe-8673-18cb57448207 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.812595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693310c-0ffa-46fe-8673-18cb57448207 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.812595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693310c-0ffa-46fe-8673-18cb57448207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.834192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d693310c-0ffa-46fe-8673-18cb57448207 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:48.835385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d693310c-0ffa-46fe-8673-18cb57448207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:48.881106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626b2bf-b25a-47b6-80b3-46b7190414fa None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:48.905936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:48.924200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262da8e6-77c6-48e8-a738-ac9e4264b289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:48.924847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262da8e6-77c6-48e8-a738-ac9e4264b289 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:48.925361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262da8e6-77c6-48e8-a738-ac9e4264b289 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:48.925492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 180/363] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:48.925572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262da8e6-77c6-48e8-a738-ac9e4264b289 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:48.927981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693310c-0ffa-46fe-8673-18cb57448207 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:48.928093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262da8e6-77c6-48e8-a738-ac9e4264b289 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:48.928093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262da8e6-77c6-48e8-a738-ac9e4264b289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:48.928093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262da8e6-77c6-48e8-a738-ac9e4264b289 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:48.941524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 184/364] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:48.942408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262da8e6-77c6-48e8-a738-ac9e4264b289 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:49.037297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:49.055789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693310c-0ffa-46fe-8673-18cb57448207 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:49.071622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262da8e6-77c6-48e8-a738-ac9e4264b289 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:49.105482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41060553-18d5-4a88-91a9-b3c7195ae793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:49.106588 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41060553-18d5-4a88-91a9-b3c7195ae793 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:49.106588 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41060553-18d5-4a88-91a9-b3c7195ae793 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:49.106751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41060553-18d5-4a88-91a9-b3c7195ae793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:49.122474 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd772d7-5047-482d-9b5a-52a3cfab166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:49.123070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd772d7-5047-482d-9b5a-52a3cfab166e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:49.123346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd772d7-5047-482d-9b5a-52a3cfab166e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:49.123639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd772d7-5047-482d-9b5a-52a3cfab166e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:49.124372 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd772d7-5047-482d-9b5a-52a3cfab166e None admin] RBAC: Adding query filter params (group.id=130ab88bb892478198b0e3e61da62fcc, role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:49.124735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd772d7-5047-482d-9b5a-52a3cfab166e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:49.127180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd772d7-5047-482d-9b5a-52a3cfab166e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:49.135623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 185/365] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:48 2022] GET /identity/v3/role_assignments?group.id=130ab88bb892478198b0e3e61da62fcc&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:49.136455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd772d7-5047-482d-9b5a-52a3cfab166e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:49.197767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41060553-18d5-4a88-91a9-b3c7195ae793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:49.199250 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41060553-18d5-4a88-91a9-b3c7195ae793 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:49.216934 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:49.291059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd772d7-5047-482d-9b5a-52a3cfab166e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:49.315011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:49.330065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 181/366] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:49.330919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41060553-18d5-4a88-91a9-b3c7195ae793 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:49.337756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d3667-b418-4547-bc92-df3eb0a43bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:49.338542 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d3667-b418-4547-bc92-df3eb0a43bcf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:49.339633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d3667-b418-4547-bc92-df3eb0a43bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:49.339921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d3667-b418-4547-bc92-df3eb0a43bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:49.340683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d3667-b418-4547-bc92-df3eb0a43bcf None admin] RBAC: Adding query filter params (role.id=17973d7fca9143289f3ca113a29e4166, user.id=7a318df8b13f4475b8d9d1a17e026a3e, scope.system=all) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:49.340988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d3667-b418-4547-bc92-df3eb0a43bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:49.343041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d3667-b418-4547-bc92-df3eb0a43bcf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:49.361576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 186/367] 10.222.0.50 () {56 vars in 1308 bytes} [Mon Aug 8 17:50:49 2022] GET /identity/v3/role_assignments?user.id=7a318df8b13f4475b8d9d1a17e026a3e&scope.system=all&role.id=17973d7fca9143289f3ca113a29e4166 => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:49.363092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d3667-b418-4547-bc92-df3eb0a43bcf None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:49.509077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d3667-b418-4547-bc92-df3eb0a43bcf None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:49.522435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:49.534428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41060553-18d5-4a88-91a9-b3c7195ae793 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:49.574074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60add8-9a1a-4599-93ba-80d7dde30177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:49.574758 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60add8-9a1a-4599-93ba-80d7dde30177 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:49.575038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60add8-9a1a-4599-93ba-80d7dde30177 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:49.575302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60add8-9a1a-4599-93ba-80d7dde30177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:49.609598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348ad18-1a16-4a0a-baaf-bb29427507af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:49.610239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348ad18-1a16-4a0a-baaf-bb29427507af None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:49.610513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348ad18-1a16-4a0a-baaf-bb29427507af None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:49.610778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348ad18-1a16-4a0a-baaf-bb29427507af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:49.625965 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6348ad18-1a16-4a0a-baaf-bb29427507af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6348ad18-1a16-4a0a-baaf-bb29427507af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:49.632156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:49.633129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:49.633947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 187/368] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:49.634618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348ad18-1a16-4a0a-baaf-bb29427507af None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:49.643264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:49.658167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60add8-9a1a-4599-93ba-80d7dde30177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:49.659641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60add8-9a1a-4599-93ba-80d7dde30177 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:49.782479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348ad18-1a16-4a0a-baaf-bb29427507af None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:49.800798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 182/369] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:49.805280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60add8-9a1a-4599-93ba-80d7dde30177 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:49.830573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167edc27-65f4-4ff1-845c-ca35d485d7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:49.831489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167edc27-65f4-4ff1-845c-ca35d485d7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:49.831994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167edc27-65f4-4ff1-845c-ca35d485d7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:49.832442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167edc27-65f4-4ff1-845c-ca35d485d7e3 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:49.834530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167edc27-65f4-4ff1-845c-ca35d485d7e3 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:49.836605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167edc27-65f4-4ff1-845c-ca35d485d7e3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:49.838904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 188/370] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:49 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:49.840136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167edc27-65f4-4ff1-845c-ca35d485d7e3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:49.900905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:49.917042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:49.935163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60add8-9a1a-4599-93ba-80d7dde30177 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:49.980916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2accbeb-7b71-4593-9054-90da6884770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:49.981646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2accbeb-7b71-4593-9054-90da6884770c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:49.981989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2accbeb-7b71-4593-9054-90da6884770c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:49.982320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2accbeb-7b71-4593-9054-90da6884770c None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:49.983098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2accbeb-7b71-4593-9054-90da6884770c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:49.983461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2accbeb-7b71-4593-9054-90da6884770c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:49.984079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167edc27-65f4-4ff1-845c-ca35d485d7e3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:49.985107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2accbeb-7b71-4593-9054-90da6884770c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:49.996279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 183/371] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:49.996279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2accbeb-7b71-4593-9054-90da6884770c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:50.041142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043471b6-ea7c-48f7-baf5-fb8f213d8077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:50.043000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043471b6-ea7c-48f7-baf5-fb8f213d8077 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.043466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043471b6-ea7c-48f7-baf5-fb8f213d8077 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.044044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043471b6-ea7c-48f7-baf5-fb8f213d8077 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.045505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043471b6-ea7c-48f7-baf5-fb8f213d8077 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:50.047205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043471b6-ea7c-48f7-baf5-fb8f213d8077 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:50.050106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 189/372] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:49 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:50.051067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043471b6-ea7c-48f7-baf5-fb8f213d8077 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:50.121059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2accbeb-7b71-4593-9054-90da6884770c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:50.163667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b158d-bf54-4836-9bd2-73a6243e5b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:50.164409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b158d-bf54-4836-9bd2-73a6243e5b60 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.164840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b158d-bf54-4836-9bd2-73a6243e5b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.165155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b158d-bf54-4836-9bd2-73a6243e5b60 None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.166366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b158d-bf54-4836-9bd2-73a6243e5b60 None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:50.169113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b158d-bf54-4836-9bd2-73a6243e5b60 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:50.169712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 184/373] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:49 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:50.171345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b158d-bf54-4836-9bd2-73a6243e5b60 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:50.192608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:50.194602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:50.198306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043471b6-ea7c-48f7-baf5-fb8f213d8077 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:50.238887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755cc40-ee57-4f83-87a5-7f76dbffb134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:50.239692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755cc40-ee57-4f83-87a5-7f76dbffb134 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.240621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755cc40-ee57-4f83-87a5-7f76dbffb134 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.241221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755cc40-ee57-4f83-87a5-7f76dbffb134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.292451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b158d-bf54-4836-9bd2-73a6243e5b60 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:50.324370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755cc40-ee57-4f83-87a5-7f76dbffb134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:50.326004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755cc40-ee57-4f83-87a5-7f76dbffb134 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:50.335908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6e197-e406-4106-a3d9-14f8369c8a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:50.336979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6e197-e406-4106-a3d9-14f8369c8a26 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.338048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6e197-e406-4106-a3d9-14f8369c8a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.338048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6e197-e406-4106-a3d9-14f8369c8a26 None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.339988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6e197-e406-4106-a3d9-14f8369c8a26 None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:50.341926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6e197-e406-4106-a3d9-14f8369c8a26 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:50.344221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 185/374] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:50 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:50.344921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6e197-e406-4106-a3d9-14f8369c8a26 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:50.456325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:50.469201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:50.473000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 190/375] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:50.475981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6e197-e406-4106-a3d9-14f8369c8a26 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:50.478906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf14e8-5bc9-4c34-93a2-14417287f0e5 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.479577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf14e8-5bc9-4c34-93a2-14417287f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.479577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf14e8-5bc9-4c34-93a2-14417287f0e5 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.481162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 191/376] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:50.481870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf14e8-5bc9-4c34-93a2-14417287f0e5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:50.524690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8693ccb-a9e5-46a2-b920-f3104d92557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:50.525299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8693ccb-a9e5-46a2-b920-f3104d92557d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.527166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8693ccb-a9e5-46a2-b920-f3104d92557d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.527166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8693ccb-a9e5-46a2-b920-f3104d92557d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.527166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8693ccb-a9e5-46a2-b920-f3104d92557d None admin] RBAC: Adding query filter params (group.id=130ab88bb892478198b0e3e61da62fcc, role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=00da5d0d955743658d1eb4ac7e00247f) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:50.527166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8693ccb-a9e5-46a2-b920-f3104d92557d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:50.528258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8693ccb-a9e5-46a2-b920-f3104d92557d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:50.534423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 186/377] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:50 2022] GET /identity/v3/role_assignments?group.id=130ab88bb892478198b0e3e61da62fcc&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:50.535224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8693ccb-a9e5-46a2-b920-f3104d92557d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:50.623393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf14e8-5bc9-4c34-93a2-14417287f0e5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:50.660961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8693ccb-a9e5-46a2-b920-f3104d92557d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:50.665672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bad3aa-4bce-4e61-b4a2-02569edc06bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:50.666608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bad3aa-4bce-4e61-b4a2-02569edc06bf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.667079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bad3aa-4bce-4e61-b4a2-02569edc06bf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.667563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bad3aa-4bce-4e61-b4a2-02569edc06bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.668491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bad3aa-4bce-4e61-b4a2-02569edc06bf None admin] RBAC: Adding query filter params (group.id=130ab88bb892478198b0e3e61da62fcc, role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=00da5d0d955743658d1eb4ac7e00247f) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:50.669101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bad3aa-4bce-4e61-b4a2-02569edc06bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:50.671409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bad3aa-4bce-4e61-b4a2-02569edc06bf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:50.684810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 192/378] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:50 2022] GET /identity/v3/role_assignments?group.id=130ab88bb892478198b0e3e61da62fcc&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:50.686955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a10bf2-a750-477a-af2a-c8aabc2f6478 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.687442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a10bf2-a750-477a-af2a-c8aabc2f6478 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.688022 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a10bf2-a750-477a-af2a-c8aabc2f6478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.700474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d6955-7b66-422b-aabf-59874aadf576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:50.701151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d6955-7b66-422b-aabf-59874aadf576 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.701428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d6955-7b66-422b-aabf-59874aadf576 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.702242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d6955-7b66-422b-aabf-59874aadf576 None admin] PATH_INFO: `/v3/projects/6c1d22d0f80a4861a903f2c834f48928` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.707617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d6955-7b66-422b-aabf-59874aadf576 None admin] RBAC: Authorizing `identity:get_project(project_id=6c1d22d0f80a4861a903f2c834f48928)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:50.707617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d6955-7b66-422b-aabf-59874aadf576 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:50.707617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 187/379] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:50 2022] GET /identity/v3/projects/6c1d22d0f80a4861a903f2c834f48928 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:50.714327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d6955-7b66-422b-aabf-59874aadf576 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:50.719193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d7a10bf2-a750-477a-af2a-c8aabc2f6478 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:50.720763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d7a10bf2-a750-477a-af2a-c8aabc2f6478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:50.739020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:50.835621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:50.850900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 193/380] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:50.877595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d6955-7b66-422b-aabf-59874aadf576 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:50.913946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a10bf2-a750-477a-af2a-c8aabc2f6478 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:50.929698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1d0a5-55d7-4624-a3bb-1fdbd842dad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:50.930346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d0a5-55d7-4624-a3bb-1fdbd842dad0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:50.930622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d0a5-55d7-4624-a3bb-1fdbd842dad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:50.930889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d0a5-55d7-4624-a3bb-1fdbd842dad0 None admin] PATH_INFO: `/v3/groups/eb7f3e34c2334904a1c31d32c8ed1260` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:50.939142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1d0a5-55d7-4624-a3bb-1fdbd842dad0 None admin] RBAC: Authorizing `identity:get_group(group_id=eb7f3e34c2334904a1c31d32c8ed1260)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:50.940627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1d0a5-55d7-4624-a3bb-1fdbd842dad0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:50.943284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 188/381] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:50 2022] GET /identity/v3/groups/eb7f3e34c2334904a1c31d32c8ed1260 => generated 229 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:50.949061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1d0a5-55d7-4624-a3bb-1fdbd842dad0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:51.044371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a10bf2-a750-477a-af2a-c8aabc2f6478 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:51.072973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:51.086566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fb3e3-2229-4156-9e2d-f5a1942c9852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:51.087200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fb3e3-2229-4156-9e2d-f5a1942c9852 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:51.087471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fb3e3-2229-4156-9e2d-f5a1942c9852 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:51.087737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fb3e3-2229-4156-9e2d-f5a1942c9852 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:51.107176 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fb3e3-2229-4156-9e2d-f5a1942c9852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-481fb3e3-2229-4156-9e2d-f5a1942c9852 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:51.111954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:51.112881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:51.113696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 194/382] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:51.120537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1d0a5-55d7-4624-a3bb-1fdbd842dad0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:51.127751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fb3e3-2229-4156-9e2d-f5a1942c9852 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:51.169122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0530c73-59e2-49ca-8069-ad3c14031c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:51.169735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0530c73-59e2-49ca-8069-ad3c14031c80 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:51.170007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0530c73-59e2-49ca-8069-ad3c14031c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:51.170277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0530c73-59e2-49ca-8069-ad3c14031c80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:51.171048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0530c73-59e2-49ca-8069-ad3c14031c80 None admin] RBAC: Adding query filter params (group.id=eb7f3e34c2334904a1c31d32c8ed1260, role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=6c1d22d0f80a4861a903f2c834f48928) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:51.171349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0530c73-59e2-49ca-8069-ad3c14031c80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:51.174632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0530c73-59e2-49ca-8069-ad3c14031c80 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:51.180800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 189/383] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:50 2022] GET /identity/v3/role_assignments?group.id=eb7f3e34c2334904a1c31d32c8ed1260&scope.project.id=6c1d22d0f80a4861a903f2c834f48928&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:51.220940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:51.270312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fb3e3-2229-4156-9e2d-f5a1942c9852 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:51.314558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc5b68-482a-4a7e-9bed-9fcb1547314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:51.315421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc5b68-482a-4a7e-9bed-9fcb1547314a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:51.315856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc5b68-482a-4a7e-9bed-9fcb1547314a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:51.316257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc5b68-482a-4a7e-9bed-9fcb1547314a None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:51.317096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc5b68-482a-4a7e-9bed-9fcb1547314a None admin] RBAC: Adding query filter params (name=member) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:51.317527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc5b68-482a-4a7e-9bed-9fcb1547314a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:51.319040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc5b68-482a-4a7e-9bed-9fcb1547314a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:51.333331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 195/384] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:51.338826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0530c73-59e2-49ca-8069-ad3c14031c80 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:51.484661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0530c73-59e2-49ca-8069-ad3c14031c80 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:51.531930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafe34c-43a7-4062-b5ab-1d9d4a2d8b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:51.532581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafe34c-43a7-4062-b5ab-1d9d4a2d8b6b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:51.532858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafe34c-43a7-4062-b5ab-1d9d4a2d8b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:51.533126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafe34c-43a7-4062-b5ab-1d9d4a2d8b6b None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:51.534253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafe34c-43a7-4062-b5ab-1d9d4a2d8b6b None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:51.535674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafe34c-43a7-4062-b5ab-1d9d4a2d8b6b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:51.537632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 190/385] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:51 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:51.542703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc5b68-482a-4a7e-9bed-9fcb1547314a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:51.581170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac41a27-f1da-4818-9f42-357f6bfed515 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:51.581371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac41a27-f1da-4818-9f42-357f6bfed515 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:51.581620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac41a27-f1da-4818-9f42-357f6bfed515 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:51.583202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 191/386] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:51.588209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72538-8b56-4aaf-92a1-92f3e91a4d13 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:51.588487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72538-8b56-4aaf-92a1-92f3e91a4d13 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:51.588799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e72538-8b56-4aaf-92a1-92f3e91a4d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:51.606610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-70e72538-8b56-4aaf-92a1-92f3e91a4d13 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:51.607784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-70e72538-8b56-4aaf-92a1-92f3e91a4d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:51.624852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:51.708755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc5b68-482a-4a7e-9bed-9fcb1547314a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:51.740028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:51.741742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 192/387] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:51.758918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ddb3a9-a72b-4e98-b0ed-736cad64c46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:51.759542 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddb3a9-a72b-4e98-b0ed-736cad64c46c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:51.759810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddb3a9-a72b-4e98-b0ed-736cad64c46c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:51.760073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddb3a9-a72b-4e98-b0ed-736cad64c46c None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:51.761205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ddb3a9-a72b-4e98-b0ed-736cad64c46c None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:51.762707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ddb3a9-a72b-4e98-b0ed-736cad64c46c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:51.764730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 196/388] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:51 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:51.770957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72538-8b56-4aaf-92a1-92f3e91a4d13 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:51.782671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ddb3a9-a72b-4e98-b0ed-736cad64c46c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:51.939790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ddb3a9-a72b-4e98-b0ed-736cad64c46c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:51.968252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e72538-8b56-4aaf-92a1-92f3e91a4d13 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:51.988208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b642f55-4f15-4b3c-b046-7e316e5fc1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:51.989169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b642f55-4f15-4b3c-b046-7e316e5fc1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:51.990437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b642f55-4f15-4b3c-b046-7e316e5fc1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:51.990952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b642f55-4f15-4b3c-b046-7e316e5fc1f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:52.013604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b642f55-4f15-4b3c-b046-7e316e5fc1f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8b642f55-4f15-4b3c-b046-7e316e5fc1f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:52.019659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:52.020599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:52.021410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:52.028544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 197/389] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:52.036528 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b642f55-4f15-4b3c-b046-7e316e5fc1f7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:52.042589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869edfd-9e74-42d7-96b4-2e346af32962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:52.043560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869edfd-9e74-42d7-96b4-2e346af32962 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:52.044042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869edfd-9e74-42d7-96b4-2e346af32962 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:52.044571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869edfd-9e74-42d7-96b4-2e346af32962 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/groups/130ab88bb892478198b0e3e61da62fcc/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:52.049856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869edfd-9e74-42d7-96b4-2e346af32962 None admin] RBAC: Authorizing `identity:create_grant(project_id=f47f695da6654f48a1df25b7f4bf15eb, group_id=130ab88bb892478198b0e3e61da62fcc, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:52.051638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869edfd-9e74-42d7-96b4-2e346af32962 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:52.089668 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:52.143763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 193/390] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 17:50:51 2022] PUT /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/groups/130ab88bb892478198b0e3e61da62fcc/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:52.145792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b642f55-4f15-4b3c-b046-7e316e5fc1f7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:52.182679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569a60e-c563-4a6c-aeac-1f1bbbb34686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:52.183282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569a60e-c563-4a6c-aeac-1f1bbbb34686 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:52.183553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569a60e-c563-4a6c-aeac-1f1bbbb34686 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:52.183817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569a60e-c563-4a6c-aeac-1f1bbbb34686 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:52.184546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569a60e-c563-4a6c-aeac-1f1bbbb34686 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:52.184848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569a60e-c563-4a6c-aeac-1f1bbbb34686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:52.186765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569a60e-c563-4a6c-aeac-1f1bbbb34686 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:52.196141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 198/391] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:52.202295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869edfd-9e74-42d7-96b4-2e346af32962 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:52.206529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:52.345729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869edfd-9e74-42d7-96b4-2e346af32962 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:52.393766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1007b2-13e9-40ad-af3b-d4d795bb0d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:52.394537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1007b2-13e9-40ad-af3b-d4d795bb0d7b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:52.394808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1007b2-13e9-40ad-af3b-d4d795bb0d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:52.395069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1007b2-13e9-40ad-af3b-d4d795bb0d7b None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:52.396151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1007b2-13e9-40ad-af3b-d4d795bb0d7b None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:52.397607 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1007b2-13e9-40ad-af3b-d4d795bb0d7b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:52.399155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 194/392] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:52 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:52.404706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569a60e-c563-4a6c-aeac-1f1bbbb34686 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:52.509274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569a60e-c563-4a6c-aeac-1f1bbbb34686 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:52.520838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:52.542918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dca9ef-f3d9-4952-935f-191646a53db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:52.543539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dca9ef-f3d9-4952-935f-191646a53db7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:52.543809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dca9ef-f3d9-4952-935f-191646a53db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:52.544071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dca9ef-f3d9-4952-935f-191646a53db7 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:52.545755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dca9ef-f3d9-4952-935f-191646a53db7 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:52.547241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dca9ef-f3d9-4952-935f-191646a53db7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:52.549135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 199/393] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:52 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:52.555068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1007b2-13e9-40ad-af3b-d4d795bb0d7b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:52.653737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:52.684812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1007b2-13e9-40ad-af3b-d4d795bb0d7b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:52.734625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a40308-d3e0-4dd9-b7c1-f05a070a13e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:52.735287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a40308-d3e0-4dd9-b7c1-f05a070a13e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:52.735552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a40308-d3e0-4dd9-b7c1-f05a070a13e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:52.735810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a40308-d3e0-4dd9-b7c1-f05a070a13e4 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/groups/130ab88bb892478198b0e3e61da62fcc/roles/b2ded61baed84402a2fa33eda9462c6f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:52.738049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a40308-d3e0-4dd9-b7c1-f05a070a13e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f47f695da6654f48a1df25b7f4bf15eb, group_id=130ab88bb892478198b0e3e61da62fcc, role_id=b2ded61baed84402a2fa33eda9462c6f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:52.739480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a40308-d3e0-4dd9-b7c1-f05a070a13e4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:52.828379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 195/394] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 17:50:52 2022] PUT /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb/groups/130ab88bb892478198b0e3e61da62fcc/roles/b2ded61baed84402a2fa33eda9462c6f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:52.964927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c030d-069b-45c9-9d5a-18accfb85d9f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:52.965273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c030d-069b-45c9-9d5a-18accfb85d9f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:52.965474 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c030d-069b-45c9-9d5a-18accfb85d9f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:52.967056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 200/395] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:52.972936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd999a-4962-4152-8d7b-92fe1c6c67ae None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:52.973214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd999a-4962-4152-8d7b-92fe1c6c67ae None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:52.973484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd999a-4962-4152-8d7b-92fe1c6c67ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:52.995313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-16cd999a-4962-4152-8d7b-92fe1c6c67ae None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:52.996494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-16cd999a-4962-4152-8d7b-92fe1c6c67ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:53.079780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 196/396] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:53.104922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c030d-069b-45c9-9d5a-18accfb85d9f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:53.230436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c030d-069b-45c9-9d5a-18accfb85d9f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:53.264961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fc613-5cf8-4de7-bbf3-cc9478a65e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:53.265584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fc613-5cf8-4de7-bbf3-cc9478a65e79 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:53.265849 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fc613-5cf8-4de7-bbf3-cc9478a65e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:53.266107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fc613-5cf8-4de7-bbf3-cc9478a65e79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:53.277970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82fc613-5cf8-4de7-bbf3-cc9478a65e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a82fc613-5cf8-4de7-bbf3-cc9478a65e79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:53.283322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:53.284214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:53.285067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 201/397] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:53.291914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd999a-4962-4152-8d7b-92fe1c6c67ae None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:53.357648 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:53.409288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2aae6-6572-48d0-8093-7f48c4eff802 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:53.409812 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2aae6-6572-48d0-8093-7f48c4eff802 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:53.409812 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2aae6-6572-48d0-8093-7f48c4eff802 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:53.411393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 202/398] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:53.411563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd999a-4962-4152-8d7b-92fe1c6c67ae None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:53.416035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6037f-d515-4c4a-a30f-7f0e3d6faa71 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:53.419529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6037f-d515-4c4a-a30f-7f0e3d6faa71 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:53.419529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6037f-d515-4c4a-a30f-7f0e3d6faa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:53.461176 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fcb6037f-d515-4c4a-a30f-7f0e3d6faa71 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:53.462361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fcb6037f-d515-4c4a-a30f-7f0e3d6faa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:53.467682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f30c-a70c-4ae4-a705-871d5291c119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:53.468290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f30c-a70c-4ae4-a705-871d5291c119 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:53.468580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f30c-a70c-4ae4-a705-871d5291c119 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:53.468843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f30c-a70c-4ae4-a705-871d5291c119 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:53.469500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323f30c-a70c-4ae4-a705-871d5291c119 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:53.469791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323f30c-a70c-4ae4-a705-871d5291c119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:53.471144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323f30c-a70c-4ae4-a705-871d5291c119 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:53.487877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 197/399] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:53.493654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f30c-a70c-4ae4-a705-871d5291c119 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:53.500862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:53.538859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 203/400] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:53.540160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206df875-db6c-45e5-bf31-b6d5e8053e68 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:53.540437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206df875-db6c-45e5-bf31-b6d5e8053e68 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:53.540730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206df875-db6c-45e5-bf31-b6d5e8053e68 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:53.544544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 204/401] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:53.547685 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4048d-5aba-4bdd-b851-4d8c5f77da03 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:53.547959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4048d-5aba-4bdd-b851-4d8c5f77da03 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:53.548218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4048d-5aba-4bdd-b851-4d8c5f77da03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:53.566319 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-06d4048d-5aba-4bdd-b851-4d8c5f77da03 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:53.567507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-06d4048d-5aba-4bdd-b851-4d8c5f77da03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:53.628587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f30c-a70c-4ae4-a705-871d5291c119 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:53.657418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 205/402] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:53.658936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4048d-5aba-4bdd-b851-4d8c5f77da03 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:53.659739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:53.663269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5a44a-2b31-4fcf-a7e2-1c137262fab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:53.663897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5a44a-2b31-4fcf-a7e2-1c137262fab2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:53.664171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5a44a-2b31-4fcf-a7e2-1c137262fab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:53.664441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5a44a-2b31-4fcf-a7e2-1c137262fab2 None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:53.665721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5a44a-2b31-4fcf-a7e2-1c137262fab2 None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:53.667271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5a44a-2b31-4fcf-a7e2-1c137262fab2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:53.669411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 198/403] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:53 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:53.675655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5a44a-2b31-4fcf-a7e2-1c137262fab2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:53.768821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:53.777101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4048d-5aba-4bdd-b851-4d8c5f77da03 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:53.809810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5a44a-2b31-4fcf-a7e2-1c137262fab2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:53.824065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6b1bf-0030-41e6-8864-2d9caa0a6414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:53.824958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b1bf-0030-41e6-8864-2d9caa0a6414 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:53.825343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b1bf-0030-41e6-8864-2d9caa0a6414 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:53.825705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b1bf-0030-41e6-8864-2d9caa0a6414 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:53.837745 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6b1bf-0030-41e6-8864-2d9caa0a6414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f2c6b1bf-0030-41e6-8864-2d9caa0a6414 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:53.842670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:53.843772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:53.844791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:53.851339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 206/404] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:53.851339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6b1bf-0030-41e6-8864-2d9caa0a6414 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:53.862458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e22e40-4b81-41d2-8644-27da82e1a075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:53.863530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e22e40-4b81-41d2-8644-27da82e1a075 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:53.863956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e22e40-4b81-41d2-8644-27da82e1a075 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:53.864403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e22e40-4b81-41d2-8644-27da82e1a075 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:53.866228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e22e40-4b81-41d2-8644-27da82e1a075 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:53.868316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e22e40-4b81-41d2-8644-27da82e1a075 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:53.876043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 199/405] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:53 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:53.876043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e22e40-4b81-41d2-8644-27da82e1a075 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:53.949728 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:53.966259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6b1bf-0030-41e6-8864-2d9caa0a6414 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:53.993508 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e22e40-4b81-41d2-8644-27da82e1a075 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:54.008268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349b3c7-ee2c-4a0f-a506-974c841748d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.009263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349b3c7-ee2c-4a0f-a506-974c841748d0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.009697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349b3c7-ee2c-4a0f-a506-974c841748d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.010151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349b3c7-ee2c-4a0f-a506-974c841748d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.023628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349b3c7-ee2c-4a0f-a506-974c841748d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8349b3c7-ee2c-4a0f-a506-974c841748d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:54.028955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:54.030004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:54.030815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:54.033479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 207/406] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:54.034485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349b3c7-ee2c-4a0f-a506-974c841748d0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:54.034989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d93cec-c51f-4795-ab85-40d7ab80c20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.035686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d93cec-c51f-4795-ab85-40d7ab80c20c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.036070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d93cec-c51f-4795-ab85-40d7ab80c20c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.036431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d93cec-c51f-4795-ab85-40d7ab80c20c None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.037209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d93cec-c51f-4795-ab85-40d7ab80c20c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:54.037598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d93cec-c51f-4795-ab85-40d7ab80c20c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:54.039273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d93cec-c51f-4795-ab85-40d7ab80c20c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:54.053723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:54.056576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 200/407] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:54.057594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d93cec-c51f-4795-ab85-40d7ab80c20c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:54.152321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349b3c7-ee2c-4a0f-a506-974c841748d0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:54.189107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d93cec-c51f-4795-ab85-40d7ab80c20c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:54.189931 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ba61d-4ccf-44d0-b01d-341f36bf61a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.190896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ba61d-4ccf-44d0-b01d-341f36bf61a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.191351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ba61d-4ccf-44d0-b01d-341f36bf61a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.191976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ba61d-4ccf-44d0-b01d-341f36bf61a4 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f/groups/130ab88bb892478198b0e3e61da62fcc/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.194117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ba61d-4ccf-44d0-b01d-341f36bf61a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=00da5d0d955743658d1eb4ac7e00247f, group_id=130ab88bb892478198b0e3e61da62fcc, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:54.195742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ba61d-4ccf-44d0-b01d-341f36bf61a4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:54.219001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:54.224702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e743678-d88c-44a3-bbd1-051460feafa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.225302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e743678-d88c-44a3-bbd1-051460feafa2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.225571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e743678-d88c-44a3-bbd1-051460feafa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.225831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e743678-d88c-44a3-bbd1-051460feafa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.226475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e743678-d88c-44a3-bbd1-051460feafa2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:54.226765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e743678-d88c-44a3-bbd1-051460feafa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:54.228122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e743678-d88c-44a3-bbd1-051460feafa2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:54.237075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 201/408] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:54.238181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e743678-d88c-44a3-bbd1-051460feafa2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:54.296887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 208/409] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 17:50:54 2022] PUT /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f/groups/130ab88bb892478198b0e3e61da62fcc/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:54.298188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ba61d-4ccf-44d0-b01d-341f36bf61a4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:54.403841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e743678-d88c-44a3-bbd1-051460feafa2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:54.447852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:54.473156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecd5b1-f4fc-4ee7-8d99-6a3a9e4e27a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.473756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecd5b1-f4fc-4ee7-8d99-6a3a9e4e27a7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecd5b1-f4fc-4ee7-8d99-6a3a9e4e27a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.474292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecd5b1-f4fc-4ee7-8d99-6a3a9e4e27a7 None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.475416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecd5b1-f4fc-4ee7-8d99-6a3a9e4e27a7 None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:54.476884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecd5b1-f4fc-4ee7-8d99-6a3a9e4e27a7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:54.478798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 202/410] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:54 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:54.485150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecd5b1-f4fc-4ee7-8d99-6a3a9e4e27a7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:54.514860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ba61d-4ccf-44d0-b01d-341f36bf61a4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:54.552638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cfb0b-76a0-40bb-9b08-97e459d06f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.553265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cfb0b-76a0-40bb-9b08-97e459d06f8a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.553534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cfb0b-76a0-40bb-9b08-97e459d06f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.553796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cfb0b-76a0-40bb-9b08-97e459d06f8a None admin] PATH_INFO: `/v3/groups/eb7f3e34c2334904a1c31d32c8ed1260` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.554967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cfb0b-76a0-40bb-9b08-97e459d06f8a None admin] RBAC: Authorizing `identity:get_group(group_id=eb7f3e34c2334904a1c31d32c8ed1260)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:54.556407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cfb0b-76a0-40bb-9b08-97e459d06f8a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:54.558303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 209/411] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:54 2022] GET /identity/v3/groups/eb7f3e34c2334904a1c31d32c8ed1260 => generated 229 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:54.559011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:54.565543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cfb0b-76a0-40bb-9b08-97e459d06f8a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:54.618494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecd5b1-f4fc-4ee7-8d99-6a3a9e4e27a7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:54.662682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf771840-ff0b-4b22-9227-5a780e84ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.663876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf771840-ff0b-4b22-9227-5a780e84ac5d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.664224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf771840-ff0b-4b22-9227-5a780e84ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.664580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf771840-ff0b-4b22-9227-5a780e84ac5d None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.665796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf771840-ff0b-4b22-9227-5a780e84ac5d None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:54.667845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf771840-ff0b-4b22-9227-5a780e84ac5d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:54.671134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 203/412] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:54 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:54.676674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf771840-ff0b-4b22-9227-5a780e84ac5d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:54.692959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cfb0b-76a0-40bb-9b08-97e459d06f8a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:54.731375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a422f2-0d83-4a16-a6ba-f4549959f5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.732414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a422f2-0d83-4a16-a6ba-f4549959f5be None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.733412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a422f2-0d83-4a16-a6ba-f4549959f5be None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.733898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a422f2-0d83-4a16-a6ba-f4549959f5be None admin] PATH_INFO: `/v3/projects/6c1d22d0f80a4861a903f2c834f48928` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.735338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a422f2-0d83-4a16-a6ba-f4549959f5be None admin] RBAC: Authorizing `identity:get_project(project_id=6c1d22d0f80a4861a903f2c834f48928)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:54.737357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a422f2-0d83-4a16-a6ba-f4549959f5be None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:54.740847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 210/413] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:54 2022] GET /identity/v3/projects/6c1d22d0f80a4861a903f2c834f48928 => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:54.742435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd4c34-6cfb-43c4-9d32-0d10f3a3ee5d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.742908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd4c34-6cfb-43c4-9d32-0d10f3a3ee5d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.743361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd4c34-6cfb-43c4-9d32-0d10f3a3ee5d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.745218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 211/414] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:54.746190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd4c34-6cfb-43c4-9d32-0d10f3a3ee5d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:54.759574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:54.779583 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf771840-ff0b-4b22-9227-5a780e84ac5d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:54.802340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:54.816492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cda932-5495-4815-b72d-9d0df589ee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.817024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cda932-5495-4815-b72d-9d0df589ee51 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.817348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cda932-5495-4815-b72d-9d0df589ee51 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.817679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cda932-5495-4815-b72d-9d0df589ee51 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f/groups/130ab88bb892478198b0e3e61da62fcc/roles/b2ded61baed84402a2fa33eda9462c6f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.819847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cda932-5495-4815-b72d-9d0df589ee51 None admin] RBAC: Authorizing `identity:create_grant(project_id=00da5d0d955743658d1eb4ac7e00247f, group_id=130ab88bb892478198b0e3e61da62fcc, role_id=b2ded61baed84402a2fa33eda9462c6f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:54.821326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cda932-5495-4815-b72d-9d0df589ee51 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:54.874484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd4c34-6cfb-43c4-9d32-0d10f3a3ee5d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:54.890243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 204/415] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 17:50:54 2022] PUT /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f/groups/130ab88bb892478198b0e3e61da62fcc/roles/b2ded61baed84402a2fa33eda9462c6f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:50:54.891241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb1daf-0dae-4f9a-b1a0-e21c4b86fa5d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.891519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb1daf-0dae-4f9a-b1a0-e21c4b86fa5d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.891808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb1daf-0dae-4f9a-b1a0-e21c4b86fa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.926036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbc200-c4d2-418f-aa41-218aada7e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:54.927098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbc200-c4d2-418f-aa41-218aada7e6be None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:54.927565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbc200-c4d2-418f-aa41-218aada7e6be None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:54.928087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbc200-c4d2-418f-aa41-218aada7e6be None admin] PATH_INFO: `/v3/projects/6c1d22d0f80a4861a903f2c834f48928/groups/eb7f3e34c2334904a1c31d32c8ed1260/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:54.931278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbc200-c4d2-418f-aa41-218aada7e6be None admin] RBAC: Authorizing `identity:create_grant(project_id=6c1d22d0f80a4861a903f2c834f48928, group_id=eb7f3e34c2334904a1c31d32c8ed1260, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:54.933241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbc200-c4d2-418f-aa41-218aada7e6be None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:54.945731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8acb1daf-0dae-4f9a-b1a0-e21c4b86fa5d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:54.946947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8acb1daf-0dae-4f9a-b1a0-e21c4b86fa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:55.003816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 212/416] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 17:50:54 2022] PUT /identity/v3/projects/6c1d22d0f80a4861a903f2c834f48928/groups/eb7f3e34c2334904a1c31d32c8ed1260/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:50:55.048224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 205/417] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:55.078981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbc200-c4d2-418f-aa41-218aada7e6be None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:55.160420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:55.194153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbc200-c4d2-418f-aa41-218aada7e6be None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:55.230123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd9af7-17f8-48a5-8816-66dfffdc07a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:55.231190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd9af7-17f8-48a5-8816-66dfffdc07a5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:55.231665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd9af7-17f8-48a5-8816-66dfffdc07a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:55.232394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd9af7-17f8-48a5-8816-66dfffdc07a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:55.243242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd9af7-17f8-48a5-8816-66dfffdc07a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-64cd9af7-17f8-48a5-8816-66dfffdc07a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:55.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:55.249211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:55.250059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:55.252749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 213/418] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:55.258266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb1daf-0dae-4f9a-b1a0-e21c4b86fa5d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:55.363817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb1daf-0dae-4f9a-b1a0-e21c4b86fa5d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:55.375970 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:55.400255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47466c71-bb10-4b18-be7d-dc5c04002f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:55.401247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47466c71-bb10-4b18-be7d-dc5c04002f70 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:55.401712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47466c71-bb10-4b18-be7d-dc5c04002f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:55.402167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47466c71-bb10-4b18-be7d-dc5c04002f70 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:55.403010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47466c71-bb10-4b18-be7d-dc5c04002f70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:55.403494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47466c71-bb10-4b18-be7d-dc5c04002f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:55.405765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47466c71-bb10-4b18-be7d-dc5c04002f70 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:55.415834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 206/419] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:55.421665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd9af7-17f8-48a5-8816-66dfffdc07a5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:55.455461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0014b80-e93c-499e-ae58-651e5ec8691b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:55.456070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0014b80-e93c-499e-ae58-651e5ec8691b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:55.456970 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0014b80-e93c-499e-ae58-651e5ec8691b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:55.459066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 207/420] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:55.464602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557e337-a52a-4604-958c-a5fd0c6b06e4 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:55.464877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557e337-a52a-4604-958c-a5fd0c6b06e4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:55.465183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557e337-a52a-4604-958c-a5fd0c6b06e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:55.501477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1557e337-a52a-4604-958c-a5fd0c6b06e4 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:55.502651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1557e337-a52a-4604-958c-a5fd0c6b06e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:55.546273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd9af7-17f8-48a5-8816-66dfffdc07a5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:55.580450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:55.586760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e69cfc-cb55-4727-ae2b-1682564fe0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:55.587794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e69cfc-cb55-4727-ae2b-1682564fe0ae None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:55.588286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e69cfc-cb55-4727-ae2b-1682564fe0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:55.588929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e69cfc-cb55-4727-ae2b-1682564fe0ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:55.647133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 208/421] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:55.677546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557e337-a52a-4604-958c-a5fd0c6b06e4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:55.678708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e69cfc-cb55-4727-ae2b-1682564fe0ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:55.680645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e69cfc-cb55-4727-ae2b-1682564fe0ae None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:55.815616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 214/422] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:55.821120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e69cfc-cb55-4727-ae2b-1682564fe0ae None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:55.833186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557e337-a52a-4604-958c-a5fd0c6b06e4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:55.843392 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:55.888654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:55.897512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8a62d-671f-46be-ad2d-dab812eef63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:55.898145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8a62d-671f-46be-ad2d-dab812eef63b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:55.898422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8a62d-671f-46be-ad2d-dab812eef63b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:55.898686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8a62d-671f-46be-ad2d-dab812eef63b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:55.916844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8a62d-671f-46be-ad2d-dab812eef63b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2ea8a62d-671f-46be-ad2d-dab812eef63b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:55.922201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:55.923110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:55.923910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:55.926739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 209/423] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:55.931113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8a62d-671f-46be-ad2d-dab812eef63b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:55.982833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e69cfc-cb55-4727-ae2b-1682564fe0ae None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:56.023091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04eeb32-687a-4b76-8362-2325d751e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:56.023714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04eeb32-687a-4b76-8362-2325d751e366 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:56.023990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04eeb32-687a-4b76-8362-2325d751e366 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:56.024328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04eeb32-687a-4b76-8362-2325d751e366 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:56.026958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04eeb32-687a-4b76-8362-2325d751e366 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:56.029308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04eeb32-687a-4b76-8362-2325d751e366 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:56.034816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 215/424] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:55 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:56.039738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04eeb32-687a-4b76-8362-2325d751e366 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:56.140162 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8a62d-671f-46be-ad2d-dab812eef63b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:56.185033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:56.191344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04eeb32-687a-4b76-8362-2325d751e366 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:56.204930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19991865-26ee-4031-9d35-52bfa303756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:56.205836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19991865-26ee-4031-9d35-52bfa303756e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:56.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19991865-26ee-4031-9d35-52bfa303756e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:56.207075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19991865-26ee-4031-9d35-52bfa303756e None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:56.207865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19991865-26ee-4031-9d35-52bfa303756e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:56.208170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19991865-26ee-4031-9d35-52bfa303756e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:56.213664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19991865-26ee-4031-9d35-52bfa303756e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:56.226495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 210/425] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:56.232107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19991865-26ee-4031-9d35-52bfa303756e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:56.239336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531367bb-4e97-4cb2-b60b-f02e78f59dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:56.240220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531367bb-4e97-4cb2-b60b-f02e78f59dcc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:56.245365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531367bb-4e97-4cb2-b60b-f02e78f59dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:56.245847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531367bb-4e97-4cb2-b60b-f02e78f59dcc None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:56.247212 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531367bb-4e97-4cb2-b60b-f02e78f59dcc None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:56.248900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531367bb-4e97-4cb2-b60b-f02e78f59dcc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:56.250682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 216/426] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:56 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:56.257612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531367bb-4e97-4cb2-b60b-f02e78f59dcc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:56.280252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:56.407830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531367bb-4e97-4cb2-b60b-f02e78f59dcc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:56.428186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19991865-26ee-4031-9d35-52bfa303756e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:56.466778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72617ede-6489-4e23-bd42-9251d2013353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:56.467873 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72617ede-6489-4e23-bd42-9251d2013353 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:56.468345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72617ede-6489-4e23-bd42-9251d2013353 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:56.468859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72617ede-6489-4e23-bd42-9251d2013353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:56.486538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b0a02-0bf9-4d31-b9d8-8ef8fb47b55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:56.487153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b0a02-0bf9-4d31-b9d8-8ef8fb47b55f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:56.487418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b0a02-0bf9-4d31-b9d8-8ef8fb47b55f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:56.487682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b0a02-0bf9-4d31-b9d8-8ef8fb47b55f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:56.488413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b0a02-0bf9-4d31-b9d8-8ef8fb47b55f None admin] RBAC: Adding query filter params (group.id=130ab88bb892478198b0e3e61da62fcc, role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:56.488757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b0a02-0bf9-4d31-b9d8-8ef8fb47b55f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:56.490181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b0a02-0bf9-4d31-b9d8-8ef8fb47b55f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:56.503881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 217/427] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:56 2022] GET /identity/v3/role_assignments?group.id=130ab88bb892478198b0e3e61da62fcc&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 611 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:56.549446 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72617ede-6489-4e23-bd42-9251d2013353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:56.551270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72617ede-6489-4e23-bd42-9251d2013353 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:56.579230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:56.662528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 211/428] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:56.668004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b0a02-0bf9-4d31-b9d8-8ef8fb47b55f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:56.719870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:56.731292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34499-d734-4b28-ad98-bc60e4590ebf None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:56.735526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34499-d734-4b28-ad98-bc60e4590ebf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:56.735526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34499-d734-4b28-ad98-bc60e4590ebf None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:56.735526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 212/429] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:56.739372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f9c45-3cdc-4b87-8233-fc03f90f89eb None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:56.739656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f9c45-3cdc-4b87-8233-fc03f90f89eb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:56.739929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f9c45-3cdc-4b87-8233-fc03f90f89eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:56.766145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-064f9c45-3cdc-4b87-8233-fc03f90f89eb None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:56.769906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-064f9c45-3cdc-4b87-8233-fc03f90f89eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:56.838055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b0a02-0bf9-4d31-b9d8-8ef8fb47b55f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:56.864762 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 213/430] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:56.883742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60635282-6300-4346-aa45-f23ca7040c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:56.884742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60635282-6300-4346-aa45-f23ca7040c07 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:56.885218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60635282-6300-4346-aa45-f23ca7040c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:56.885681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60635282-6300-4346-aa45-f23ca7040c07 None admin] PATH_INFO: `/v3/projects/f47f695da6654f48a1df25b7f4bf15eb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:56.888012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60635282-6300-4346-aa45-f23ca7040c07 None admin] RBAC: Authorizing `identity:get_project(project_id=f47f695da6654f48a1df25b7f4bf15eb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:56.889203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f9c45-3cdc-4b87-8233-fc03f90f89eb None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:56.889986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60635282-6300-4346-aa45-f23ca7040c07 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:56.892529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 218/431] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:56 2022] GET /identity/v3/projects/f47f695da6654f48a1df25b7f4bf15eb => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:56.898515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60635282-6300-4346-aa45-f23ca7040c07 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:56.965173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:57.013507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:57.015139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f9c45-3cdc-4b87-8233-fc03f90f89eb None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:57.018789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60635282-6300-4346-aa45-f23ca7040c07 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:57.056928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb8b6d-56d7-439e-8223-481f0fb27eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:57.057549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb8b6d-56d7-439e-8223-481f0fb27eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.057822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb8b6d-56d7-439e-8223-481f0fb27eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.058089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb8b6d-56d7-439e-8223-481f0fb27eb1 None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.059223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8e2f9-55cc-494d-ad75-cdd223534b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:57.060075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8e2f9-55cc-494d-ad75-cdd223534b82 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.060257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb8b6d-56d7-439e-8223-481f0fb27eb1 None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:57.061831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8e2f9-55cc-494d-ad75-cdd223534b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.062305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb8b6d-56d7-439e-8223-481f0fb27eb1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:57.062563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8e2f9-55cc-494d-ad75-cdd223534b82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.064486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 219/432] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:56 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:57.070184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb8b6d-56d7-439e-8223-481f0fb27eb1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:57.075546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8e2f9-55cc-494d-ad75-cdd223534b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9fe8e2f9-55cc-494d-ad75-cdd223534b82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:57.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:57.082138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 17:50:57.083024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:50:57.089150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 214/433] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:50:57.092834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8e2f9-55cc-494d-ad75-cdd223534b82 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:57.205406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb8b6d-56d7-439e-8223-481f0fb27eb1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:57.227776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8e2f9-55cc-494d-ad75-cdd223534b82 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:57.245724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:57.256084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62458d0d-a274-4afd-bf3e-7cf4a57fa677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:57.257142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458d0d-a274-4afd-bf3e-7cf4a57fa677 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.263132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458d0d-a274-4afd-bf3e-7cf4a57fa677 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.263132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458d0d-a274-4afd-bf3e-7cf4a57fa677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.263132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62458d0d-a274-4afd-bf3e-7cf4a57fa677 None admin] RBAC: Adding query filter params (group.id=130ab88bb892478198b0e3e61da62fcc, role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=f47f695da6654f48a1df25b7f4bf15eb) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:57.263132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62458d0d-a274-4afd-bf3e-7cf4a57fa677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:57.263132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62458d0d-a274-4afd-bf3e-7cf4a57fa677 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:57.268443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26e10d-a05d-4783-b4a8-1609b8e519f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:57.276547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 220/434] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:57 2022] GET /identity/v3/role_assignments?group.id=130ab88bb892478198b0e3e61da62fcc&scope.project.id=f47f695da6654f48a1df25b7f4bf15eb&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:57.276767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26e10d-a05d-4783-b4a8-1609b8e519f0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.276767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26e10d-a05d-4783-b4a8-1609b8e519f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.276767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26e10d-a05d-4783-b4a8-1609b8e519f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.276767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26e10d-a05d-4783-b4a8-1609b8e519f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:57.276767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26e10d-a05d-4783-b4a8-1609b8e519f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:57.276767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26e10d-a05d-4783-b4a8-1609b8e519f0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:57.291473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 215/435] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:50:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:57.293138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62458d0d-a274-4afd-bf3e-7cf4a57fa677 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:57.337871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:57.418094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62458d0d-a274-4afd-bf3e-7cf4a57fa677 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:57.462354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8d33e-e108-45cd-b743-1ca493766f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:57.463089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8d33e-e108-45cd-b743-1ca493766f30 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.463365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8d33e-e108-45cd-b743-1ca493766f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.463630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8d33e-e108-45cd-b743-1ca493766f30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.483574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51686531-9b5c-475b-9876-28dabb4891e8 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.484068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51686531-9b5c-475b-9876-28dabb4891e8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.484156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51686531-9b5c-475b-9876-28dabb4891e8 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.485871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 216/436] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:50:57.496831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445286b-afb7-4137-9af1-d9ab40419960 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.497445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445286b-afb7-4137-9af1-d9ab40419960 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.497938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445286b-afb7-4137-9af1-d9ab40419960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.523799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b445286b-afb7-4137-9af1-d9ab40419960 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:57.525098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b445286b-afb7-4137-9af1-d9ab40419960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:57.550426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8d33e-e108-45cd-b743-1ca493766f30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:57.551861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8d33e-e108-45cd-b743-1ca493766f30 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:57.587380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:57.620454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 217/437] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:50:57.621962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980327-d002-4cd1-b424-15c76c610d4e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.621962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980327-d002-4cd1-b424-15c76c610d4e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.622172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980327-d002-4cd1-b424-15c76c610d4e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.630180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 218/438] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:50:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:50:57.635983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefe421-fbb1-42ee-afb8-5919cbe3a265 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.636268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefe421-fbb1-42ee-afb8-5919cbe3a265 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.636585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefe421-fbb1-42ee-afb8-5919cbe3a265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.656365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-efefe421-fbb1-42ee-afb8-5919cbe3a265 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:50:57.657638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-efefe421-fbb1-42ee-afb8-5919cbe3a265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:50:57.672929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 221/439] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:57.674447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8d33e-e108-45cd-b743-1ca493766f30 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:57.738432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 219/440] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:50:57.739199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefe421-fbb1-42ee-afb8-5919cbe3a265 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:57.774209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:57.786724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8d33e-e108-45cd-b743-1ca493766f30 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:57.814522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:57.819766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31a9d5-415e-4617-baa7-544cca651218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:57.820383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31a9d5-415e-4617-baa7-544cca651218 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.820684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31a9d5-415e-4617-baa7-544cca651218 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.820950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31a9d5-415e-4617-baa7-544cca651218 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.831421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31a9d5-415e-4617-baa7-544cca651218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-0a31a9d5-415e-4617-baa7-544cca651218 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:57.836104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:57.837088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:57.837885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 222/441] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:50:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:57.839509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31a9d5-415e-4617-baa7-544cca651218 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:57.840985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefe421-fbb1-42ee-afb8-5919cbe3a265 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:57.876210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc2dd8-b0d0-4d68-a399-86fcf2ff0b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:57.876856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc2dd8-b0d0-4d68-a399-86fcf2ff0b11 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.877135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc2dd8-b0d0-4d68-a399-86fcf2ff0b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.877406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc2dd8-b0d0-4d68-a399-86fcf2ff0b11 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.878514 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc2dd8-b0d0-4d68-a399-86fcf2ff0b11 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:57.879941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc2dd8-b0d0-4d68-a399-86fcf2ff0b11 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:57.881582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 220/442] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:57 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:57.882322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc2dd8-b0d0-4d68-a399-86fcf2ff0b11 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:57.938158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31a9d5-415e-4617-baa7-544cca651218 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:57.972983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663eac7c-fd7b-4f98-88f2-c2e1a9aa761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:57.973591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663eac7c-fd7b-4f98-88f2-c2e1a9aa761b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:57.973862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663eac7c-fd7b-4f98-88f2-c2e1a9aa761b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:57.974126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663eac7c-fd7b-4f98-88f2-c2e1a9aa761b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:57.983805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663eac7c-fd7b-4f98-88f2-c2e1a9aa761b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:50:57.987213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc2dd8-b0d0-4d68-a399-86fcf2ff0b11 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-663eac7c-fd7b-4f98-88f2-c2e1a9aa761b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:50:57.988396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:50:57.989361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:50:57.990183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 223/443] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:50:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:50:57.991652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:57.993282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663eac7c-fd7b-4f98-88f2-c2e1a9aa761b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.021470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef65d-a778-4240-97d5-6bf3cc152a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.022061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef65d-a778-4240-97d5-6bf3cc152a6c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.022333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef65d-a778-4240-97d5-6bf3cc152a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.022597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ef65d-a778-4240-97d5-6bf3cc152a6c None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.023249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef65d-a778-4240-97d5-6bf3cc152a6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:58.023544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef65d-a778-4240-97d5-6bf3cc152a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.024984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ef65d-a778-4240-97d5-6bf3cc152a6c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.033018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 221/444] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:50:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:58.033794 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef65d-a778-4240-97d5-6bf3cc152a6c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.041790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:58.093359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663eac7c-fd7b-4f98-88f2-c2e1a9aa761b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:58.130370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388a939-1bb5-4957-898d-299a6b4f2fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.130990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388a939-1bb5-4957-898d-299a6b4f2fce None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.131260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388a939-1bb5-4957-898d-299a6b4f2fce None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.131529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388a939-1bb5-4957-898d-299a6b4f2fce None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.133052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388a939-1bb5-4957-898d-299a6b4f2fce None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.133740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ef65d-a778-4240-97d5-6bf3cc152a6c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:58.134496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388a939-1bb5-4957-898d-299a6b4f2fce None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.136052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 224/445] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:57 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:58.136844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388a939-1bb5-4957-898d-299a6b4f2fce None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.167971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b9082-5d46-43c1-a7a0-2d708669152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.168594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b9082-5d46-43c1-a7a0-2d708669152c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.168868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b9082-5d46-43c1-a7a0-2d708669152c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.169132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b9082-5d46-43c1-a7a0-2d708669152c None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.169790 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b9082-5d46-43c1-a7a0-2d708669152c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:58.170140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b9082-5d46-43c1-a7a0-2d708669152c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.171483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b9082-5d46-43c1-a7a0-2d708669152c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.179265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 222/446] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:58.180019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b9082-5d46-43c1-a7a0-2d708669152c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.211141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:58.244029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388a939-1bb5-4957-898d-299a6b4f2fce None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:58.261005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:58.277351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5403c-9292-4140-9b29-b3447d037f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.277952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5403c-9292-4140-9b29-b3447d037f88 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.278225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5403c-9292-4140-9b29-b3447d037f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.278555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5403c-9292-4140-9b29-b3447d037f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.291477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b9082-5d46-43c1-a7a0-2d708669152c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:58.329293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8ce8-9740-407c-ad19-af2f5114f26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.330150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8ce8-9740-407c-ad19-af2f5114f26d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.330424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8ce8-9740-407c-ad19-af2f5114f26d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.330683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8ce8-9740-407c-ad19-af2f5114f26d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.331411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f8ce8-9740-407c-ad19-af2f5114f26d None admin] RBAC: Adding query filter params (group.id=130ab88bb892478198b0e3e61da62fcc, role.id=f61a519ee1a74ddeb7cb809ee43d9825, scope.project.id=00da5d0d955743658d1eb4ac7e00247f) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:58.331712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f8ce8-9740-407c-ad19-af2f5114f26d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.333960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f8ce8-9740-407c-ad19-af2f5114f26d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.354704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5403c-9292-4140-9b29-b3447d037f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.356356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5403c-9292-4140-9b29-b3447d037f88 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.356891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 223/447] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:58 2022] GET /identity/v3/role_assignments?group.id=130ab88bb892478198b0e3e61da62fcc&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=f61a519ee1a74ddeb7cb809ee43d9825 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:58.358197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8ce8-9740-407c-ad19-af2f5114f26d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.487035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:58.498152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 225/448] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:58.501789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5403c-9292-4140-9b29-b3447d037f88 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.505015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8ce8-9740-407c-ad19-af2f5114f26d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:58.534777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:58.554344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3cf52-7e19-4ce9-9f6b-548b1031fce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.557397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3cf52-7e19-4ce9-9f6b-548b1031fce7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.558849 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3cf52-7e19-4ce9-9f6b-548b1031fce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.559885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3cf52-7e19-4ce9-9f6b-548b1031fce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.641001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5403c-9292-4140-9b29-b3447d037f88 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:58.644277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3cf52-7e19-4ce9-9f6b-548b1031fce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.645928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3cf52-7e19-4ce9-9f6b-548b1031fce7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.680152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a2dc2-08a3-4988-b4ef-559b841886b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.680818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a2dc2-08a3-4988-b4ef-559b841886b2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.681077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a2dc2-08a3-4988-b4ef-559b841886b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.681337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a2dc2-08a3-4988-b4ef-559b841886b2 None admin] PATH_INFO: `/v3/projects/00da5d0d955743658d1eb4ac7e00247f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.682460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a2dc2-08a3-4988-b4ef-559b841886b2 None admin] RBAC: Authorizing `identity:get_project(project_id=00da5d0d955743658d1eb4ac7e00247f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.684021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a2dc2-08a3-4988-b4ef-559b841886b2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.686010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 226/449] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:58 2022] GET /identity/v3/projects/00da5d0d955743658d1eb4ac7e00247f => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:58.692762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a2dc2-08a3-4988-b4ef-559b841886b2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.744419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:58.753530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 224/450] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:50:58.758956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3cf52-7e19-4ce9-9f6b-548b1031fce7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.777705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:58.793271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a2dc2-08a3-4988-b4ef-559b841886b2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:58.828210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782e459-a3b4-40c5-9e0a-5bbf6fe37acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.828858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782e459-a3b4-40c5-9e0a-5bbf6fe37acb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.829130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782e459-a3b4-40c5-9e0a-5bbf6fe37acb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.829653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782e459-a3b4-40c5-9e0a-5bbf6fe37acb None admin] PATH_INFO: `/v3/groups/130ab88bb892478198b0e3e61da62fcc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.830764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782e459-a3b4-40c5-9e0a-5bbf6fe37acb None admin] RBAC: Authorizing `identity:get_group(group_id=130ab88bb892478198b0e3e61da62fcc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.832205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782e459-a3b4-40c5-9e0a-5bbf6fe37acb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.833794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 227/451] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:58 2022] GET /identity/v3/groups/130ab88bb892478198b0e3e61da62fcc => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:58.839412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782e459-a3b4-40c5-9e0a-5bbf6fe37acb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.858624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3cf52-7e19-4ce9-9f6b-548b1031fce7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:58.890950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904facc0-e5c0-4f56-bb22-313a2e463489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.891566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904facc0-e5c0-4f56-bb22-313a2e463489 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.891842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904facc0-e5c0-4f56-bb22-313a2e463489 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.892112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904facc0-e5c0-4f56-bb22-313a2e463489 None admin] PATH_INFO: `/v3/projects/6c1d22d0f80a4861a903f2c834f48928` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.893257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904facc0-e5c0-4f56-bb22-313a2e463489 None admin] RBAC: Authorizing `identity:get_project(project_id=6c1d22d0f80a4861a903f2c834f48928)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.894703 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904facc0-e5c0-4f56-bb22-313a2e463489 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.896461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 225/452] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:50:58 2022] GET /identity/v3/projects/6c1d22d0f80a4861a903f2c834f48928 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:50:58.901622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904facc0-e5c0-4f56-bb22-313a2e463489 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:58.936960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782e459-a3b4-40c5-9e0a-5bbf6fe37acb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:58.959784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:58.972971 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe197f26-d346-41c2-ad76-54d00e7cf5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:58.974002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197f26-d346-41c2-ad76-54d00e7cf5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:58.974456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197f26-d346-41c2-ad76-54d00e7cf5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:58.974731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197f26-d346-41c2-ad76-54d00e7cf5b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:58.976496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe197f26-d346-41c2-ad76-54d00e7cf5b4 None admin] RBAC: Adding query filter params (group.id=130ab88bb892478198b0e3e61da62fcc, role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=00da5d0d955743658d1eb4ac7e00247f) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:58.976831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe197f26-d346-41c2-ad76-54d00e7cf5b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:58.979003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe197f26-d346-41c2-ad76-54d00e7cf5b4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:58.989005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 228/453] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:58 2022] GET /identity/v3/role_assignments?group.id=130ab88bb892478198b0e3e61da62fcc&scope.project.id=00da5d0d955743658d1eb4ac7e00247f&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:59.007050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904facc0-e5c0-4f56-bb22-313a2e463489 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:59.045933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e8edd-d488-4ccc-9a88-4c7a60618a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:59.046576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e8edd-d488-4ccc-9a88-4c7a60618a31 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:59.046848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e8edd-d488-4ccc-9a88-4c7a60618a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:59.047111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e8edd-d488-4ccc-9a88-4c7a60618a31 None admin] PATH_INFO: `/v3/groups/eb7f3e34c2334904a1c31d32c8ed1260` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:59.048310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e8edd-d488-4ccc-9a88-4c7a60618a31 None admin] RBAC: Authorizing `identity:get_group(group_id=eb7f3e34c2334904a1c31d32c8ed1260)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:59.049832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e8edd-d488-4ccc-9a88-4c7a60618a31 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:59.051478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 226/454] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 17:50:58 2022] GET /identity/v3/groups/eb7f3e34c2334904a1c31d32c8ed1260 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:50:59.057622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe197f26-d346-41c2-ad76-54d00e7cf5b4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:50:59.078648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:50:59.162125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe197f26-d346-41c2-ad76-54d00e7cf5b4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:50:59.195184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c8f94-09d0-471b-80d9-1199257daedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:50:59.195771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c8f94-09d0-471b-80d9-1199257daedf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:50:59.196039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c8f94-09d0-471b-80d9-1199257daedf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:50:59.196298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c8f94-09d0-471b-80d9-1199257daedf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:50:59.197040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c8f94-09d0-471b-80d9-1199257daedf None admin] RBAC: Adding query filter params (group.id=eb7f3e34c2334904a1c31d32c8ed1260, role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=6c1d22d0f80a4861a903f2c834f48928) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:50:59.197335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c8f94-09d0-471b-80d9-1199257daedf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:50:59.198732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c8f94-09d0-471b-80d9-1199257daedf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:50:59.205222 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 229/455] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 17:50:59 2022] GET /identity/v3/role_assignments?group.id=eb7f3e34c2334904a1c31d32c8ed1260&scope.project.id=6c1d22d0f80a4861a903f2c834f48928&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:01.680999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465ef90-213f-44b3-9d32-a61bdd11d364 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:01.681542 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465ef90-213f-44b3-9d32-a61bdd11d364 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:01.681542 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465ef90-213f-44b3-9d32-a61bdd11d364 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:01.683958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 227/456] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:01.689297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd56006-80af-4599-958b-614eaa912f57 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:01.689743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd56006-80af-4599-958b-614eaa912f57 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:01.689836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd56006-80af-4599-958b-614eaa912f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:01.713524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fdd56006-80af-4599-958b-614eaa912f57 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:01.714782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fdd56006-80af-4599-958b-614eaa912f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:01.799009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 230/457] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:01.839155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465ef90-213f-44b3-9d32-a61bdd11d364 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:01.866252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb7714-993d-48b0-9c22-5266551d35c5 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:01.866583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb7714-993d-48b0-9c22-5266551d35c5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:01.866781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb7714-993d-48b0-9c22-5266551d35c5 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:01.868099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 231/458] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:01.874763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bddd4-20c4-4dd8-8d1a-87946b4c7ba9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:01.875304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bddd4-20c4-4dd8-8d1a-87946b4c7ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:01.875980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bddd4-20c4-4dd8-8d1a-87946b4c7ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:01.907932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-981bddd4-20c4-4dd8-8d1a-87946b4c7ba9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:01.909147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-981bddd4-20c4-4dd8-8d1a-87946b4c7ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:01.922131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:02.009620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465ef90-213f-44b3-9d32-a61bdd11d364 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:02.031408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 232/459] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:51:02.064125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:02.072743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774780b-1f84-434a-be1c-3dae659c0123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:02.073371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774780b-1f84-434a-be1c-3dae659c0123 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.073649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774780b-1f84-434a-be1c-3dae659c0123 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.073909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774780b-1f84-434a-be1c-3dae659c0123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.088032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bddd4-20c4-4dd8-8d1a-87946b4c7ba9 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:02.088206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774780b-1f84-434a-be1c-3dae659c0123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:02.089714 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774780b-1f84-434a-be1c-3dae659c0123 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b774780b-1f84-434a-be1c-3dae659c0123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:02.111243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:02.112233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:02.113506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:02.113506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:02.113506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:02.115333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 228/460] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:02.122287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774780b-1f84-434a-be1c-3dae659c0123 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:02.231111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774780b-1f84-434a-be1c-3dae659c0123 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:02.233420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bddd4-20c4-4dd8-8d1a-87946b4c7ba9 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:02.267892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cb0b8-3ac1-4833-82ee-e0d72d075c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:02.268410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cb0b8-3ac1-4833-82ee-e0d72d075c3b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.268706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cb0b8-3ac1-4833-82ee-e0d72d075c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.268894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d4ad0-48f7-4fdd-b431-6098332c6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:02.269002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cb0b8-3ac1-4833-82ee-e0d72d075c3b None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.269512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d4ad0-48f7-4fdd-b431-6098332c6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.269607 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cb0b8-3ac1-4833-82ee-e0d72d075c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:02.269785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d4ad0-48f7-4fdd-b431-6098332c6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.269877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cb0b8-3ac1-4833-82ee-e0d72d075c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:02.270051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d4ad0-48f7-4fdd-b431-6098332c6cf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.271183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cb0b8-3ac1-4833-82ee-e0d72d075c3b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:02.280038 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d4ad0-48f7-4fdd-b431-6098332c6cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:02.281918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d4ad0-48f7-4fdd-b431-6098332c6cf6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:02.283526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 229/461] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:02.283920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:02.284724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da54d58-5df6-48a2-bcff-882a399d2afa None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.284993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da54d58-5df6-48a2-bcff-882a399d2afa None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.285346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da54d58-5df6-48a2-bcff-882a399d2afa None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.290478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 230/462] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:02.290478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57c3d4-7725-4877-b4e1-0b545b1e38fa None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.290478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57c3d4-7725-4877-b4e1-0b545b1e38fa None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.290478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57c3d4-7725-4877-b4e1-0b545b1e38fa None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.290478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 231/463] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:02.290478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57c3d4-7725-4877-b4e1-0b545b1e38fa None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6c2d4ad0-48f7-4fdd-b431-6098332c6cf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:02.306025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:02.306914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:02.307801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:02.307801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:02.307801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:02.307801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 233/464] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:02.308596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8fa17-dc98-449c-8fe1-b8720677212e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.308810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8fa17-dc98-449c-8fe1-b8720677212e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.309070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8fa17-dc98-449c-8fe1-b8720677212e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.327562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f7e8fa17-dc98-449c-8fe1-b8720677212e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:02.328740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f7e8fa17-dc98-449c-8fe1-b8720677212e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:02.344248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:02.397463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 234/465] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:02.398579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237060e-7df5-4f9c-aabf-41a3566e50da None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.398859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237060e-7df5-4f9c-aabf-41a3566e50da None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.399120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237060e-7df5-4f9c-aabf-41a3566e50da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.410545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57c3d4-7725-4877-b4e1-0b545b1e38fa None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:02.416493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1237060e-7df5-4f9c-aabf-41a3566e50da None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:02.417719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1237060e-7df5-4f9c-aabf-41a3566e50da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:02.448939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a98250-5c23-445e-90ca-4cdc9953e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:02.449798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a98250-5c23-445e-90ca-4cdc9953e084 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.450070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a98250-5c23-445e-90ca-4cdc9953e084 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.450333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a98250-5c23-445e-90ca-4cdc9953e084 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.450986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a98250-5c23-445e-90ca-4cdc9953e084 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:02.452433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a98250-5c23-445e-90ca-4cdc9953e084 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:02.487276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 235/466] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:02.488098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237060e-7df5-4f9c-aabf-41a3566e50da None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:02.543153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:02.568720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 232/467] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:51:02 2022] POST /identity/v3/users => generated 246 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:02.568720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a98250-5c23-445e-90ca-4cdc9953e084 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:02.598769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237060e-7df5-4f9c-aabf-41a3566e50da None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:02.641468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87076aa-2131-4f22-b31d-4a7e9bfcf19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:02.642083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87076aa-2131-4f22-b31d-4a7e9bfcf19d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.642355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87076aa-2131-4f22-b31d-4a7e9bfcf19d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.642616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87076aa-2131-4f22-b31d-4a7e9bfcf19d None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.643267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87076aa-2131-4f22-b31d-4a7e9bfcf19d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:02.643560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87076aa-2131-4f22-b31d-4a7e9bfcf19d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:02.645058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87076aa-2131-4f22-b31d-4a7e9bfcf19d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:02.659171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 236/468] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:02.660018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87076aa-2131-4f22-b31d-4a7e9bfcf19d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:02.719149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:02.727891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a98250-5c23-445e-90ca-4cdc9953e084 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:02.772015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff41eb7-bdb1-429d-9651-be2b03fea76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:02.772700 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41eb7-bdb1-429d-9651-be2b03fea76a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.773133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41eb7-bdb1-429d-9651-be2b03fea76a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.773278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41eb7-bdb1-429d-9651-be2b03fea76a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.786411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff41eb7-bdb1-429d-9651-be2b03fea76a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:02.789988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff41eb7-bdb1-429d-9651-be2b03fea76a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:02.790105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87076aa-2131-4f22-b31d-4a7e9bfcf19d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-7ff41eb7-bdb1-429d-9651-be2b03fea76a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:02.805290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:02.806286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:02.807148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:02.807148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:02.807148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:02.807148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 233/469] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:02.807148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff41eb7-bdb1-429d-9651-be2b03fea76a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:02.810610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:02.838685 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43c07b-1d74-404c-9c78-c636bbe22990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:02.839311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43c07b-1d74-404c-9c78-c636bbe22990 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.839589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43c07b-1d74-404c-9c78-c636bbe22990 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.839843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43c07b-1d74-404c-9c78-c636bbe22990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.856229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43c07b-1d74-404c-9c78-c636bbe22990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:02.857846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43c07b-1d74-404c-9c78-c636bbe22990 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3b43c07b-1d74-404c-9c78-c636bbe22990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:02.873365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:02.874298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:02.875164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:02.875164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:02.875164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:02.875164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 237/470] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:02.875164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43c07b-1d74-404c-9c78-c636bbe22990 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:02.932902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff41eb7-bdb1-429d-9651-be2b03fea76a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:02.970714 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea18aa4-2d04-4248-8f2f-9d40abb20a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:02.971324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea18aa4-2d04-4248-8f2f-9d40abb20a1e None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:02.971599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea18aa4-2d04-4248-8f2f-9d40abb20a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:02.971879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea18aa4-2d04-4248-8f2f-9d40abb20a1e None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:02.972521 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea18aa4-2d04-4248-8f2f-9d40abb20a1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:02.973963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea18aa4-2d04-4248-8f2f-9d40abb20a1e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:03.003172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43c07b-1d74-404c-9c78-c636bbe22990 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:03.038745 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5dd59-1b6c-425f-84e1-3e1a7eb132e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:03.039340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5dd59-1b6c-425f-84e1-3e1a7eb132e5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:03.039615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5dd59-1b6c-425f-84e1-3e1a7eb132e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:03.039881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5dd59-1b6c-425f-84e1-3e1a7eb132e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:03.040551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5dd59-1b6c-425f-84e1-3e1a7eb132e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:03.040847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5dd59-1b6c-425f-84e1-3e1a7eb132e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:03.042342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5dd59-1b6c-425f-84e1-3e1a7eb132e5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:03.056028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 234/471] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:51:02 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:03.056028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea18aa4-2d04-4248-8f2f-9d40abb20a1e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:03.060784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 238/472] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:03.061194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:03.066098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5dd59-1b6c-425f-84e1-3e1a7eb132e5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:03.108185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:03.190782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea18aa4-2d04-4248-8f2f-9d40abb20a1e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:03.207769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5dd59-1b6c-425f-84e1-3e1a7eb132e5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:03.226597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a47bd6-1eaa-4275-a07c-57c0fc0ffdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:03.227265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a47bd6-1eaa-4275-a07c-57c0fc0ffdbc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:03.227546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a47bd6-1eaa-4275-a07c-57c0fc0ffdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:03.227805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a47bd6-1eaa-4275-a07c-57c0fc0ffdbc None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:03.228455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a47bd6-1eaa-4275-a07c-57c0fc0ffdbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:03.228763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a47bd6-1eaa-4275-a07c-57c0fc0ffdbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:03.230144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a47bd6-1eaa-4275-a07c-57c0fc0ffdbc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:03.244671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 235/473] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:03.245957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e740c-28ed-45c4-ba3f-82334563f2bf None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:03.246235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e740c-28ed-45c4-ba3f-82334563f2bf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:03.246512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e740c-28ed-45c4-ba3f-82334563f2bf None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:03.247693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 236/474] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:03.253468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109402e8-c77f-4c02-ba73-7de440f28bfe None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:03.253740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109402e8-c77f-4c02-ba73-7de440f28bfe None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:03.254010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109402e8-c77f-4c02-ba73-7de440f28bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:03.259018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb061e-294d-40c3-bce2-56dc1fbea4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:03.259886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb061e-294d-40c3-bce2-56dc1fbea4e5 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:03.260536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb061e-294d-40c3-bce2-56dc1fbea4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:03.261020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb061e-294d-40c3-bce2-56dc1fbea4e5 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:03.262899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb061e-294d-40c3-bce2-56dc1fbea4e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:03.265408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb061e-294d-40c3-bce2-56dc1fbea4e5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:03.274244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-109402e8-c77f-4c02-ba73-7de440f28bfe None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:03.275329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-109402e8-c77f-4c02-ba73-7de440f28bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:03.340663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:03.348791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 237/475] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:51:03.359154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 239/476] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:51:03 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:51:03.359424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109402e8-c77f-4c02-ba73-7de440f28bfe None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:03.381664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb061e-294d-40c3-bce2-56dc1fbea4e5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:03.540104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109402e8-c77f-4c02-ba73-7de440f28bfe None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:03.551004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:03.574609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb061e-294d-40c3-bce2-56dc1fbea4e5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:03.583254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced2bc1-e470-420e-8fe4-b44f1932d27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:03.583946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced2bc1-e470-420e-8fe4-b44f1932d27d None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:03.584300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced2bc1-e470-420e-8fe4-b44f1932d27d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:03.584655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced2bc1-e470-420e-8fe4-b44f1932d27d None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:03.585355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced2bc1-e470-420e-8fe4-b44f1932d27d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:03.586803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced2bc1-e470-420e-8fe4-b44f1932d27d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:03.616497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:03.617058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:03.617328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:03.617587 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:03.618237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:03.619594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:03.672914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.sql.core [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 17:51:03.672914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 17:51:03.672914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [parameters: {'id': '0e2a6783e69447c4b8f9c31af476a5a0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 17:51:03.672914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80401) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 17:51:03.678989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 238/477] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:51:03 2022] POST /identity/v3/users => generated 249 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application conn.query(q) Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result.read() Aug 08 17:51:03.690444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 17:51:03.691458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application next(self.gen) Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application yield resource Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application next(self.gen) Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise self.value Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application session.commit() Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 17:51:03.692429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.session.flush() Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application compat.raise_( Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 17:51:03.693363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 17:51:03.694295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application conn.query(q) Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application result.read() Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 17:51:03.695216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [parameters: {'id': '0e2a6783e69447c4b8f9c31af476a5a0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:03.696009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 17:51:03.698552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:03.700437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 240/478] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 17:51:03 2022] POST /identity/v3/roles => generated 144 bytes in 311 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:03.707383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced2bc1-e470-420e-8fe4-b44f1932d27d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:03.761847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:03.817417 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced2bc1-e470-420e-8fe4-b44f1932d27d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:03.877494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0433-704d-44d6-92dc-4713ca85047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:03.878061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0433-704d-44d6-92dc-4713ca85047a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:03.878336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0433-704d-44d6-92dc-4713ca85047a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:03.878601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0433-704d-44d6-92dc-4713ca85047a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:03.894525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd0433-704d-44d6-92dc-4713ca85047a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5bdd0433-704d-44d6-92dc-4713ca85047a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:03.899811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:03.900756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 17:51:03.901577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:03.904868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 239/479] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:03.907445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:04.021647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb859ea-0bff-4e41-9679-ad7d96f08691 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:04.058787 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596ebff-1fcb-4210-99f0-d7d9bc634944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:04.059440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596ebff-1fcb-4210-99f0-d7d9bc634944 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:04.059715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596ebff-1fcb-4210-99f0-d7d9bc634944 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:04.059975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596ebff-1fcb-4210-99f0-d7d9bc634944 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:04.060697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596ebff-1fcb-4210-99f0-d7d9bc634944 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:04.060997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596ebff-1fcb-4210-99f0-d7d9bc634944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:04.062353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596ebff-1fcb-4210-99f0-d7d9bc634944 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:04.070970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 241/480] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:05.043504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f0933-e7ac-4015-bb2e-e519116b3489 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:05.044005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f0933-e7ac-4015-bb2e-e519116b3489 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:05.044005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f0933-e7ac-4015-bb2e-e519116b3489 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:05.046136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 240/481] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:05.051551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fc3bb-a984-4440-927f-1e88cf44a54e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:05.051938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fc3bb-a984-4440-927f-1e88cf44a54e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:05.052023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fc3bb-a984-4440-927f-1e88cf44a54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:05.077436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-698fc3bb-a984-4440-927f-1e88cf44a54e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:05.078610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-698fc3bb-a984-4440-927f-1e88cf44a54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:05.080922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:05.165130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 242/482] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:51:05.204847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f0933-e7ac-4015-bb2e-e519116b3489 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:05.337497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f0933-e7ac-4015-bb2e-e519116b3489 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:05.348254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:05.377024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf167f4-4e3f-4879-b8dd-c58747b9e7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:05.377645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf167f4-4e3f-4879-b8dd-c58747b9e7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:05.377914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf167f4-4e3f-4879-b8dd-c58747b9e7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:05.378262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf167f4-4e3f-4879-b8dd-c58747b9e7b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:05.389582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf167f4-4e3f-4879-b8dd-c58747b9e7b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5cf167f4-4e3f-4879-b8dd-c58747b9e7b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:05.394276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:05.395151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:05.395958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 241/483] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:05.401409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fc3bb-a984-4440-927f-1e88cf44a54e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:05.520540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fc3bb-a984-4440-927f-1e88cf44a54e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:05.566248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd06c51-066a-41ab-8f4b-dd9a5d03b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:05.566893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd06c51-066a-41ab-8f4b-dd9a5d03b520 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:05.567168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd06c51-066a-41ab-8f4b-dd9a5d03b520 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:05.567431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd06c51-066a-41ab-8f4b-dd9a5d03b520 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:05.568086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd06c51-066a-41ab-8f4b-dd9a5d03b520 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:05.568377 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd06c51-066a-41ab-8f4b-dd9a5d03b520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:05.569764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd06c51-066a-41ab-8f4b-dd9a5d03b520 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:05.576397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:05.583923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 243/484] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:05.589499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf167f4-4e3f-4879-b8dd-c58747b9e7b4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:05.662165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1efc8-b572-4fe5-abdf-393bfba076f4 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:05.662505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1efc8-b572-4fe5-abdf-393bfba076f4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:05.662711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1efc8-b572-4fe5-abdf-393bfba076f4 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:05.664284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 244/485] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:05.669185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032c68a-ee1a-40ac-83e6-29ad6f46ca58 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:05.669468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032c68a-ee1a-40ac-83e6-29ad6f46ca58 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:05.670347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032c68a-ee1a-40ac-83e6-29ad6f46ca58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:05.699541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d032c68a-ee1a-40ac-83e6-29ad6f46ca58 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:05.702419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d032c68a-ee1a-40ac-83e6-29ad6f46ca58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:05.796253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf167f4-4e3f-4879-b8dd-c58747b9e7b4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:05.800053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 245/486] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:51:05.846311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd839869-489d-4619-9295-4b7d95b410a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:05.846929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd839869-489d-4619-9295-4b7d95b410a0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:05.847203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd839869-489d-4619-9295-4b7d95b410a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:05.847472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd839869-489d-4619-9295-4b7d95b410a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:05.859104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd839869-489d-4619-9295-4b7d95b410a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:05.860656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd839869-489d-4619-9295-4b7d95b410a0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:05.866221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032c68a-ee1a-40ac-83e6-29ad6f46ca58 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-cd839869-489d-4619-9295-4b7d95b410a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:05.891459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:05.892538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:05.893390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:05.893390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:05.893390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:05.895121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 242/487] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:05.903090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd839869-489d-4619-9295-4b7d95b410a0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:05.938159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:06.004299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032c68a-ee1a-40ac-83e6-29ad6f46ca58 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.017200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd839869-489d-4619-9295-4b7d95b410a0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.038593 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b564c-1648-428c-b4f0-4895c6d2958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.039217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b564c-1648-428c-b4f0-4895c6d2958c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.039492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b564c-1648-428c-b4f0-4895c6d2958c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.039757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b564c-1648-428c-b4f0-4895c6d2958c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.044864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:06.050603 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b564c-1648-428c-b4f0-4895c6d2958c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:06.052986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b20dd-09a2-445f-90f1-aa29727a65b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.053642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b20dd-09a2-445f-90f1-aa29727a65b3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.053919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b20dd-09a2-445f-90f1-aa29727a65b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.054186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b20dd-09a2-445f-90f1-aa29727a65b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.054832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b20dd-09a2-445f-90f1-aa29727a65b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:06.055130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b20dd-09a2-445f-90f1-aa29727a65b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-491b564c-1648-428c-b4f0-4895c6d2958c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:06.055500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:06.056661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:06.057455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:06.058082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b20dd-09a2-445f-90f1-aa29727a65b3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:06.058160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 246/488] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:06.058727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bebecb-7ecc-464c-9919-047b76f51328 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.063441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bebecb-7ecc-464c-9919-047b76f51328 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.063441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bebecb-7ecc-464c-9919-047b76f51328 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.063441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 247/489] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:06.064881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bebecb-7ecc-464c-9919-047b76f51328 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:06.070585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 243/490] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:06.076955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0407db-f2c1-40d3-bdb5-d934fddd1cc5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.077340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0407db-f2c1-40d3-bdb5-d934fddd1cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.077705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0407db-f2c1-40d3-bdb5-d934fddd1cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.097723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bf0407db-f2c1-40d3-bdb5-d934fddd1cc5 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:06.098998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bf0407db-f2c1-40d3-bdb5-d934fddd1cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:06.169452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 244/491] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:06.171172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0407db-f2c1-40d3-bdb5-d934fddd1cc5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:06.187687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:06.191369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bebecb-7ecc-464c-9919-047b76f51328 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.237009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a876987-a2e5-4d6e-abd0-0f59a6ccfd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.238068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876987-a2e5-4d6e-abd0-0f59a6ccfd68 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.238464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876987-a2e5-4d6e-abd0-0f59a6ccfd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.238740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876987-a2e5-4d6e-abd0-0f59a6ccfd68 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.239408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a876987-a2e5-4d6e-abd0-0f59a6ccfd68 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:06.239852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a876987-a2e5-4d6e-abd0-0f59a6ccfd68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:06.241546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a876987-a2e5-4d6e-abd0-0f59a6ccfd68 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:06.255795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 248/492] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:06.256954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a876987-a2e5-4d6e-abd0-0f59a6ccfd68 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:06.293887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0407db-f2c1-40d3-bdb5-d934fddd1cc5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.319277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:06.331073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133560c2-515b-4297-b372-231be00e7667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.331694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133560c2-515b-4297-b372-231be00e7667 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.331966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133560c2-515b-4297-b372-231be00e7667 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.332224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133560c2-515b-4297-b372-231be00e7667 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.339792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133560c2-515b-4297-b372-231be00e7667 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:06.341267 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133560c2-515b-4297-b372-231be00e7667 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-133560c2-515b-4297-b372-231be00e7667 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:06.353991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:06.355082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 245/493] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 17:51:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:06.355993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133560c2-515b-4297-b372-231be00e7667 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:06.371364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a876987-a2e5-4d6e-abd0-0f59a6ccfd68 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.408217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412c6ce-b1fa-4c8f-a259-174ceb8b0903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.408872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412c6ce-b1fa-4c8f-a259-174ceb8b0903 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.409145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412c6ce-b1fa-4c8f-a259-174ceb8b0903 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.409408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412c6ce-b1fa-4c8f-a259-174ceb8b0903 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.420622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412c6ce-b1fa-4c8f-a259-174ceb8b0903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d412c6ce-b1fa-4c8f-a259-174ceb8b0903 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:06.425950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:06.426885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:06.427728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 249/494] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:06.428409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97853ac5-35c3-4e33-ad94-e1fb9131820e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.429127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97853ac5-35c3-4e33-ad94-e1fb9131820e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.429127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97853ac5-35c3-4e33-ad94-e1fb9131820e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.434181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 250/495] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:06.434181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97853ac5-35c3-4e33-ad94-e1fb9131820e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:06.435619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:06.467856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133560c2-515b-4297-b372-231be00e7667 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.501629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4366e-4b3d-414a-a04e-8dc6021aa3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.502234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4366e-4b3d-414a-a04e-8dc6021aa3ef None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.502511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4366e-4b3d-414a-a04e-8dc6021aa3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.502776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4366e-4b3d-414a-a04e-8dc6021aa3ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.513457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4366e-4b3d-414a-a04e-8dc6021aa3ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:06.514911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4366e-4b3d-414a-a04e-8dc6021aa3ef None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-7de4366e-4b3d-414a-a04e-8dc6021aa3ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:06.529124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:06.530303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:06.531183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:06.531183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:06.531183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:06.531183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 246/496] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:06.531183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4366e-4b3d-414a-a04e-8dc6021aa3ef None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:06.549700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97853ac5-35c3-4e33-ad94-e1fb9131820e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.564553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:06.588742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ab3ac-cbae-4552-b887-141653351852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.589336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ab3ac-cbae-4552-b887-141653351852 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.589900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ab3ac-cbae-4552-b887-141653351852 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.589900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ab3ac-cbae-4552-b887-141653351852 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.590545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ab3ac-cbae-4552-b887-141653351852 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:06.590860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ab3ac-cbae-4552-b887-141653351852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:06.592300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ab3ac-cbae-4552-b887-141653351852 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:06.631465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 251/497] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 17:51:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:06.632608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393850ea-69cb-4213-acfd-818ec4d3224d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.632890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393850ea-69cb-4213-acfd-818ec4d3224d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.633155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393850ea-69cb-4213-acfd-818ec4d3224d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.648327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4366e-4b3d-414a-a04e-8dc6021aa3ef None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.652411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-393850ea-69cb-4213-acfd-818ec4d3224d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:06.653615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-393850ea-69cb-4213-acfd-818ec4d3224d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:06.672853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:06.684443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a91486-074f-4285-9ed1-8e57a3ada38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.687084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a91486-074f-4285-9ed1-8e57a3ada38b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.687084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a91486-074f-4285-9ed1-8e57a3ada38b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.687084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a91486-074f-4285-9ed1-8e57a3ada38b None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.687084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a91486-074f-4285-9ed1-8e57a3ada38b None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:06.687084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a91486-074f-4285-9ed1-8e57a3ada38b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:06.688582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a91486-074f-4285-9ed1-8e57a3ada38b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:06.700487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 247/498] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:06.703310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a91486-074f-4285-9ed1-8e57a3ada38b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:06.724385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 252/499] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:06.725246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393850ea-69cb-4213-acfd-818ec4d3224d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:06.801832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a91486-074f-4285-9ed1-8e57a3ada38b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.834963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:06.836531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e73e1-df47-4849-b7f4-a5c102a89d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.837124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e73e1-df47-4849-b7f4-a5c102a89d66 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.837401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e73e1-df47-4849-b7f4-a5c102a89d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.837672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e73e1-df47-4849-b7f4-a5c102a89d66 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.837762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393850ea-69cb-4213-acfd-818ec4d3224d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.838317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e73e1-df47-4849-b7f4-a5c102a89d66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:06.838632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e73e1-df47-4849-b7f4-a5c102a89d66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:06.839993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e73e1-df47-4849-b7f4-a5c102a89d66 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:06.851449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 248/500] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:06.852632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cc37d-e963-4034-97d4-ad7c10e550bd None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.852916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cc37d-e963-4034-97d4-ad7c10e550bd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.853188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cc37d-e963-4034-97d4-ad7c10e550bd None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.854371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 249/501] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:06.856660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cc37d-e963-4034-97d4-ad7c10e550bd None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:06.877424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad86b40-1df7-4b39-8fb7-c46f5d3215bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.878018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad86b40-1df7-4b39-8fb7-c46f5d3215bc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.878296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad86b40-1df7-4b39-8fb7-c46f5d3215bc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.878562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad86b40-1df7-4b39-8fb7-c46f5d3215bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:06.901792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:06.945023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad86b40-1df7-4b39-8fb7-c46f5d3215bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:06.946491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad86b40-1df7-4b39-8fb7-c46f5d3215bc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:06.954509 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cc37d-e963-4034-97d4-ad7c10e550bd None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:06.990507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17019849-3e6e-4bbe-8793-00958cd16902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:06.991125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17019849-3e6e-4bbe-8793-00958cd16902 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:06.991391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17019849-3e6e-4bbe-8793-00958cd16902 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:06.991651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17019849-3e6e-4bbe-8793-00958cd16902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.005103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17019849-3e6e-4bbe-8793-00958cd16902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.006562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17019849-3e6e-4bbe-8793-00958cd16902 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-17019849-3e6e-4bbe-8793-00958cd16902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:07.020312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:07.021423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:07.022310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:07.022310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:07.022310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:07.022310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 250/502] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:07.022526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17019849-3e6e-4bbe-8793-00958cd16902 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.044024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:07.048240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 253/503] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:07.049133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad86b40-1df7-4b39-8fb7-c46f5d3215bc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.118682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:07.122814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17019849-3e6e-4bbe-8793-00958cd16902 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.144382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad86b40-1df7-4b39-8fb7-c46f5d3215bc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.156101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25eee9-17a7-40ce-90ef-f491175031a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.156696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25eee9-17a7-40ce-90ef-f491175031a4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.156973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25eee9-17a7-40ce-90ef-f491175031a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.157238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25eee9-17a7-40ce-90ef-f491175031a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.167190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25eee9-17a7-40ce-90ef-f491175031a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-da25eee9-17a7-40ce-90ef-f491175031a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:07.172837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:07.173939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:07.174766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 251/504] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:07.175436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e2f47-fff2-43aa-b98a-872a994d254c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.175436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e2f47-fff2-43aa-b98a-872a994d254c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.175674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e2f47-fff2-43aa-b98a-872a994d254c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.181384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fb337-a69b-469c-93d7-b922fdf912b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.182048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fb337-a69b-469c-93d7-b922fdf912b3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.182321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fb337-a69b-469c-93d7-b922fdf912b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.182608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fb337-a69b-469c-93d7-b922fdf912b3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.189665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fb337-a69b-469c-93d7-b922fdf912b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.191170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fb337-a69b-469c-93d7-b922fdf912b3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.192568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9f6e2f47-fff2-43aa-b98a-872a994d254c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:07.193779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9f6e2f47-fff2-43aa-b98a-872a994d254c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3a1fb337-a69b-469c-93d7-b922fdf912b3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:07.204472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:07.205426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 254/505] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:07.206332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fb337-a69b-469c-93d7-b922fdf912b3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.262326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 252/506] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:07.263144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e2f47-fff2-43aa-b98a-872a994d254c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.265387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:07.306999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fb337-a69b-469c-93d7-b922fdf912b3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.340836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd67fdf-013a-4f15-8548-c8dba1e5d443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.341484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd67fdf-013a-4f15-8548-c8dba1e5d443 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.342267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd67fdf-013a-4f15-8548-c8dba1e5d443 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.342533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd67fdf-013a-4f15-8548-c8dba1e5d443 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.343162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd67fdf-013a-4f15-8548-c8dba1e5d443 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:07.343456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd67fdf-013a-4f15-8548-c8dba1e5d443 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.344924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd67fdf-013a-4f15-8548-c8dba1e5d443 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.351408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:07.357114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 255/507] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:07.357916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd67fdf-013a-4f15-8548-c8dba1e5d443 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.364011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e2f47-fff2-43aa-b98a-872a994d254c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.397474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8d44d-76df-4b9e-b89e-3a95d5ed2501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.398082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8d44d-76df-4b9e-b89e-3a95d5ed2501 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.398357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8d44d-76df-4b9e-b89e-3a95d5ed2501 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.398626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8d44d-76df-4b9e-b89e-3a95d5ed2501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.409325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8d44d-76df-4b9e-b89e-3a95d5ed2501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.410705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8d44d-76df-4b9e-b89e-3a95d5ed2501 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-41c8d44d-76df-4b9e-b89e-3a95d5ed2501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:07.425281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:07.426208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:07.427065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:07.427065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:07.427065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:07.427065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 253/508] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:07.427065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8d44d-76df-4b9e-b89e-3a95d5ed2501 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.461141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd67fdf-013a-4f15-8548-c8dba1e5d443 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.480448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:07.494565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1097881-0a0f-4bfb-817d-a9b9a06d6b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.495164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097881-0a0f-4bfb-817d-a9b9a06d6b93 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.495438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097881-0a0f-4bfb-817d-a9b9a06d6b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.495706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097881-0a0f-4bfb-817d-a9b9a06d6b93 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.496357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1097881-0a0f-4bfb-817d-a9b9a06d6b93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:07.496683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1097881-0a0f-4bfb-817d-a9b9a06d6b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.498664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1097881-0a0f-4bfb-817d-a9b9a06d6b93 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.506502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 256/509] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:07.507409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1097881-0a0f-4bfb-817d-a9b9a06d6b93 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.535277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8d44d-76df-4b9e-b89e-3a95d5ed2501 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.569179 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fda7b-831f-4b49-8443-f5bc47ea15e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.569771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fda7b-831f-4b49-8443-f5bc47ea15e6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.570044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fda7b-831f-4b49-8443-f5bc47ea15e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.570310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fda7b-831f-4b49-8443-f5bc47ea15e6 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.571024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fda7b-831f-4b49-8443-f5bc47ea15e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:07.571322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fda7b-831f-4b49-8443-f5bc47ea15e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.572848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fda7b-831f-4b49-8443-f5bc47ea15e6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.598041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:07.605558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1097881-0a0f-4bfb-817d-a9b9a06d6b93 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.609909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 254/510] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:07.614595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fda7b-831f-4b49-8443-f5bc47ea15e6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.641404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944383b-b69c-4792-a3ed-23b214175a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.642027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944383b-b69c-4792-a3ed-23b214175a62 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.642300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944383b-b69c-4792-a3ed-23b214175a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.642568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944383b-b69c-4792-a3ed-23b214175a62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.652702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944383b-b69c-4792-a3ed-23b214175a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.654620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944383b-b69c-4792-a3ed-23b214175a62 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-2944383b-b69c-4792-a3ed-23b214175a62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:07.668192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:07.669158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:07.670579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:07.670579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:07.670579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:07.670579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 257/511] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:07.670579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944383b-b69c-4792-a3ed-23b214175a62 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.706590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:07.711544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fda7b-831f-4b49-8443-f5bc47ea15e6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.745325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c035e-98b6-4a71-8411-1bbabd08b607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.745937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c035e-98b6-4a71-8411-1bbabd08b607 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.746211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c035e-98b6-4a71-8411-1bbabd08b607 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.746479 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c035e-98b6-4a71-8411-1bbabd08b607 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.753365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c035e-98b6-4a71-8411-1bbabd08b607 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.754805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c035e-98b6-4a71-8411-1bbabd08b607 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-0c3c035e-98b6-4a71-8411-1bbabd08b607 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:07.765981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:07.766853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 255/512] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:07.767766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c035e-98b6-4a71-8411-1bbabd08b607 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.774585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944383b-b69c-4792-a3ed-23b214175a62 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.808700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5a323-6f44-4747-9ed8-58a28919f531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.809298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5a323-6f44-4747-9ed8-58a28919f531 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.809572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5a323-6f44-4747-9ed8-58a28919f531 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.809840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5a323-6f44-4747-9ed8-58a28919f531 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.810466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5a323-6f44-4747-9ed8-58a28919f531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:07.810758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5a323-6f44-4747-9ed8-58a28919f531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.812114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5a323-6f44-4747-9ed8-58a28919f531 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.823472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 258/513] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:07.824258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5a323-6f44-4747-9ed8-58a28919f531 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.833644 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:07.875809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c035e-98b6-4a71-8411-1bbabd08b607 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.908883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703fd22-f24c-45fc-942e-6be400c349c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.909522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703fd22-f24c-45fc-942e-6be400c349c6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.909799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703fd22-f24c-45fc-942e-6be400c349c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.910069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703fd22-f24c-45fc-942e-6be400c349c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:07.916215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:07.924399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703fd22-f24c-45fc-942e-6be400c349c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:07.924399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703fd22-f24c-45fc-942e-6be400c349c6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:07.924719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5a323-6f44-4747-9ed8-58a28919f531 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-7703fd22-f24c-45fc-942e-6be400c349c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:07.936817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:07.937712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:07.938581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:07.938581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:07.938581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:07.938581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 256/514] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:07.938804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703fd22-f24c-45fc-942e-6be400c349c6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:07.959174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136137d-65ec-4595-ba4f-bca28c5d934d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:07.959782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136137d-65ec-4595-ba4f-bca28c5d934d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:07.960054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136137d-65ec-4595-ba4f-bca28c5d934d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:07.960319 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136137d-65ec-4595-ba4f-bca28c5d934d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.027502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136137d-65ec-4595-ba4f-bca28c5d934d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.029547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136137d-65ec-4595-ba4f-bca28c5d934d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.036534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703fd22-f24c-45fc-942e-6be400c349c6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:08.046830 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:08.069821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d6d7a-607f-406e-817a-8555ee84ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:08.070699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d6d7a-607f-406e-817a-8555ee84ee13 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:08.071021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d6d7a-607f-406e-817a-8555ee84ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:08.071486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d6d7a-607f-406e-817a-8555ee84ee13 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.072310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d6d7a-607f-406e-817a-8555ee84ee13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:08.072937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d6d7a-607f-406e-817a-8555ee84ee13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.074307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d6d7a-607f-406e-817a-8555ee84ee13 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.087635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 257/515] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:08.088343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d6d7a-607f-406e-817a-8555ee84ee13 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.121687 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:08.130920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 259/516] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:08.131729 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136137d-65ec-4595-ba4f-bca28c5d934d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.190936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d6d7a-607f-406e-817a-8555ee84ee13 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:08.225225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80202a1b-ff81-419e-8ce5-b721e098b918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:08.225820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80202a1b-ff81-419e-8ce5-b721e098b918 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:08.226093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80202a1b-ff81-419e-8ce5-b721e098b918 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:08.226365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80202a1b-ff81-419e-8ce5-b721e098b918 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.227085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80202a1b-ff81-419e-8ce5-b721e098b918 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:08.227385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80202a1b-ff81-419e-8ce5-b721e098b918 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.228841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80202a1b-ff81-419e-8ce5-b721e098b918 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.229172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136137d-65ec-4595-ba4f-bca28c5d934d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:08.245832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 258/517] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 17:51:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:08.246636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80202a1b-ff81-419e-8ce5-b721e098b918 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.263029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb9a3d-91b3-4d65-acd7-9a01f7984678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:08.264154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb9a3d-91b3-4d65-acd7-9a01f7984678 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:08.264418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb9a3d-91b3-4d65-acd7-9a01f7984678 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:08.264710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb9a3d-91b3-4d65-acd7-9a01f7984678 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.271260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb9a3d-91b3-4d65-acd7-9a01f7984678 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.273067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb9a3d-91b3-4d65-acd7-9a01f7984678 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.274394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-0cbb9a3d-91b3-4d65-acd7-9a01f7984678 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:08.284565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:08.290911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 260/518] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:08.292005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb9a3d-91b3-4d65-acd7-9a01f7984678 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.340274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:08.345913 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80202a1b-ff81-419e-8ce5-b721e098b918 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:08.382476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc59eca-85f8-4a96-a8f1-d6b3e0f2bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:08.383066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc59eca-85f8-4a96-a8f1-d6b3e0f2bed9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:08.383344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc59eca-85f8-4a96-a8f1-d6b3e0f2bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:08.383604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc59eca-85f8-4a96-a8f1-d6b3e0f2bed9 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.384231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc59eca-85f8-4a96-a8f1-d6b3e0f2bed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:08.384558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc59eca-85f8-4a96-a8f1-d6b3e0f2bed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.385920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc59eca-85f8-4a96-a8f1-d6b3e0f2bed9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.386386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb9a3d-91b3-4d65-acd7-9a01f7984678 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:08.396887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 259/519] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:08.397744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc59eca-85f8-4a96-a8f1-d6b3e0f2bed9 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.422567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e12dd5-b6e9-43f2-a0d9-ae3a06138e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:08.423160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e12dd5-b6e9-43f2-a0d9-ae3a06138e28 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:08.423439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e12dd5-b6e9-43f2-a0d9-ae3a06138e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:08.423701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e12dd5-b6e9-43f2-a0d9-ae3a06138e28 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.424324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e12dd5-b6e9-43f2-a0d9-ae3a06138e28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.425816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e12dd5-b6e9-43f2-a0d9-ae3a06138e28 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.487493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:08.487853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 261/520] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:51:08 2022] POST /identity/v3/users => generated 247 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:08.488698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e12dd5-b6e9-43f2-a0d9-ae3a06138e28 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.500757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc59eca-85f8-4a96-a8f1-d6b3e0f2bed9 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:08.539497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023d417-786b-4387-8190-2cad1021fa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:08.540152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d417-786b-4387-8190-2cad1021fa89 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:08.540427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d417-786b-4387-8190-2cad1021fa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:08.540726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d417-786b-4387-8190-2cad1021fa89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.552394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023d417-786b-4387-8190-2cad1021fa89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.553954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023d417-786b-4387-8190-2cad1021fa89 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b023d417-786b-4387-8190-2cad1021fa89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:08.570156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:08.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:08.572255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:08.572255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:08.572255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:08.572255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 260/521] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:08.572712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023d417-786b-4387-8190-2cad1021fa89 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.621208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e12dd5-b6e9-43f2-a0d9-ae3a06138e28 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:08.633254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:08.672701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b010d1-baf3-40a1-92e4-f010fab48d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:08.673320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b010d1-baf3-40a1-92e4-f010fab48d38 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:08.673608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b010d1-baf3-40a1-92e4-f010fab48d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:08.673875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b010d1-baf3-40a1-92e4-f010fab48d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.697852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023d417-786b-4387-8190-2cad1021fa89 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:08.737173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d894b-5f14-432b-86cc-55e4ee2bcc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:08.737953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d894b-5f14-432b-86cc-55e4ee2bcc96 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:08.738171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d894b-5f14-432b-86cc-55e4ee2bcc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:08.738581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d894b-5f14-432b-86cc-55e4ee2bcc96 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.739289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d894b-5f14-432b-86cc-55e4ee2bcc96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:08.739542 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d894b-5f14-432b-86cc-55e4ee2bcc96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.741129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d894b-5f14-432b-86cc-55e4ee2bcc96 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.747647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b010d1-baf3-40a1-92e4-f010fab48d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.749191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b010d1-baf3-40a1-92e4-f010fab48d38 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.757804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:08.767493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 261/522] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:08.768192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d894b-5f14-432b-86cc-55e4ee2bcc96 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.856157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 262/523] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:08.857048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b010d1-baf3-40a1-92e4-f010fab48d38 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.869472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:08.890893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d894b-5f14-432b-86cc-55e4ee2bcc96 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:08.925268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c261a-2400-4ccc-9458-6694d847ae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:08.925881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c261a-2400-4ccc-9458-6694d847ae09 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:08.926147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c261a-2400-4ccc-9458-6694d847ae09 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:08.926405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c261a-2400-4ccc-9458-6694d847ae09 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:08.933394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c261a-2400-4ccc-9458-6694d847ae09 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:08.934823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c261a-2400-4ccc-9458-6694d847ae09 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-867c261a-2400-4ccc-9458-6694d847ae09 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:08.950304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:08.952646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 262/524] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:08.953641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c261a-2400-4ccc-9458-6694d847ae09 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:08.971376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b010d1-baf3-40a1-92e4-f010fab48d38 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.008206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46632d36-2fd8-40ac-bc29-9ac01d121f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.008756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46632d36-2fd8-40ac-bc29-9ac01d121f48 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.009029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46632d36-2fd8-40ac-bc29-9ac01d121f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.009293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46632d36-2fd8-40ac-bc29-9ac01d121f48 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.009921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46632d36-2fd8-40ac-bc29-9ac01d121f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:09.010214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46632d36-2fd8-40ac-bc29-9ac01d121f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.011584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46632d36-2fd8-40ac-bc29-9ac01d121f48 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.020617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:09.023906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 263/525] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:09.028428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46632d36-2fd8-40ac-bc29-9ac01d121f48 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:09.062767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c261a-2400-4ccc-9458-6694d847ae09 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.098960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9f1f2-6e53-4a0c-90d1-175658c9ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.099557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9f1f2-6e53-4a0c-90d1-175658c9ce1d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.099827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9f1f2-6e53-4a0c-90d1-175658c9ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.100095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9f1f2-6e53-4a0c-90d1-175658c9ce1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.100838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9f1f2-6e53-4a0c-90d1-175658c9ce1d None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=c496da229db34f90a4b3fdcfa582dca8) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:09.101138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9f1f2-6e53-4a0c-90d1-175658c9ce1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.102548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9f1f2-6e53-4a0c-90d1-175658c9ce1d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.103443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:09.108036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 263/526] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:08 2022] GET /identity/v3/role_assignments?user.id=c496da229db34f90a4b3fdcfa582dca8&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:09.109469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9f1f2-6e53-4a0c-90d1-175658c9ce1d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:09.139184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46632d36-2fd8-40ac-bc29-9ac01d121f48 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.179864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4804b-e916-4996-940f-b4d775ce0dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.180523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4804b-e916-4996-940f-b4d775ce0dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.180806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4804b-e916-4996-940f-b4d775ce0dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.181076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4804b-e916-4996-940f-b4d775ce0dc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.194899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4804b-e916-4996-940f-b4d775ce0dc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.196437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4804b-e916-4996-940f-b4d775ce0dc9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d8e4804b-e916-4996-940f-b4d775ce0dc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:09.213539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:09.214671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:09.215540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:09.215540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:09.215540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:09.215540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 264/527] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:09.220537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4804b-e916-4996-940f-b4d775ce0dc9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:09.269239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9f1f2-6e53-4a0c-90d1-175658c9ce1d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.296170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:09.307568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b66fda-872d-4f5e-9f5f-3f15734412fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.308432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66fda-872d-4f5e-9f5f-3f15734412fe None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.308911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66fda-872d-4f5e-9f5f-3f15734412fe None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.311241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66fda-872d-4f5e-9f5f-3f15734412fe None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.311241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b66fda-872d-4f5e-9f5f-3f15734412fe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:09.311241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b66fda-872d-4f5e-9f5f-3f15734412fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.312868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b66fda-872d-4f5e-9f5f-3f15734412fe None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.331688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 264/528] 10.222.0.50 () {56 vars in 1139 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:09.332774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b66fda-872d-4f5e-9f5f-3f15734412fe None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:09.349020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4804b-e916-4996-940f-b4d775ce0dc9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.376729 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:09.389348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec214670-9af6-4c3e-8945-83afe059d76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.390061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec214670-9af6-4c3e-8945-83afe059d76b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.391282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec214670-9af6-4c3e-8945-83afe059d76b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.391282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec214670-9af6-4c3e-8945-83afe059d76b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.398165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec214670-9af6-4c3e-8945-83afe059d76b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.399790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec214670-9af6-4c3e-8945-83afe059d76b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ec214670-9af6-4c3e-8945-83afe059d76b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:09.412249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:09.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:09.414361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:09.414361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:09.414361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:09.414361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:09.414361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:09.414361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:09.414361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:09.414361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:09.416462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 265/529] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:09.417401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec214670-9af6-4c3e-8945-83afe059d76b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:09.460639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b66fda-872d-4f5e-9f5f-3f15734412fe None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.504346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4c7c8-9995-4dd2-bd54-3d6dc2672ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.505352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4c7c8-9995-4dd2-bd54-3d6dc2672ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.506097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4c7c8-9995-4dd2-bd54-3d6dc2672ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.506506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4c7c8-9995-4dd2-bd54-3d6dc2672ac4 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.507617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4c7c8-9995-4dd2-bd54-3d6dc2672ac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:09.508055 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4c7c8-9995-4dd2-bd54-3d6dc2672ac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.509749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4c7c8-9995-4dd2-bd54-3d6dc2672ac4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.524098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 265/530] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:09.525521 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4c7c8-9995-4dd2-bd54-3d6dc2672ac4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:09.554127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec214670-9af6-4c3e-8945-83afe059d76b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.581895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:09.601305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec5da7-e480-469b-85b2-a47250c6b8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.602321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec5da7-e480-469b-85b2-a47250c6b8ca None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.602699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec5da7-e480-469b-85b2-a47250c6b8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.603217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec5da7-e480-469b-85b2-a47250c6b8ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.650545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4c7c8-9995-4dd2-bd54-3d6dc2672ac4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.664012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:09.687161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec5da7-e480-469b-85b2-a47250c6b8ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.687679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d3c66-be80-43b6-9e9c-619554064dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.688340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d3c66-be80-43b6-9e9c-619554064dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.688658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d3c66-be80-43b6-9e9c-619554064dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.688931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d3c66-be80-43b6-9e9c-619554064dc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.689619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d3c66-be80-43b6-9e9c-619554064dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:09.689920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d3c66-be80-43b6-9e9c-619554064dc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.690624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec5da7-e480-469b-85b2-a47250c6b8ca None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.691346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d3c66-be80-43b6-9e9c-619554064dc9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.708727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 266/531] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:09.708727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d3c66-be80-43b6-9e9c-619554064dc9 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:09.813449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d3c66-be80-43b6-9e9c-619554064dc9 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.814615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 266/532] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:09.815056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:09.816866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec5da7-e480-469b-85b2-a47250c6b8ca None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:09.849207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ce105-07c5-42fd-a250-937b92c4623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.849855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ce105-07c5-42fd-a250-937b92c4623a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.850100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ce105-07c5-42fd-a250-937b92c4623a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.850371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ce105-07c5-42fd-a250-937b92c4623a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.857844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ce105-07c5-42fd-a250-937b92c4623a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.859283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ce105-07c5-42fd-a250-937b92c4623a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-939ce105-07c5-42fd-a250-937b92c4623a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:09.871492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:09.872404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:09.873341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:09.873341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:09.873341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:09.873341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:09.873341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:09.873341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:09.873341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:09.873341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:09.873341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 267/533] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:09.873816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ce105-07c5-42fd-a250-937b92c4623a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:09.912314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:09.933353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec5da7-e480-469b-85b2-a47250c6b8ca None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.972035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aa4e7-f68c-4b78-b090-89855b4838ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:09.973095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aa4e7-f68c-4b78-b090-89855b4838ba None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:09.973573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aa4e7-f68c-4b78-b090-89855b4838ba None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:09.974035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aa4e7-f68c-4b78-b090-89855b4838ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:09.974798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ce105-07c5-42fd-a250-937b92c4623a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:09.975169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aa4e7-f68c-4b78-b090-89855b4838ba None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=0e95d295337b4472be946d252ac3bd7e) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:09.975688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aa4e7-f68c-4b78-b090-89855b4838ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:09.977567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aa4e7-f68c-4b78-b090-89855b4838ba None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:09.990330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 267/534] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/role_assignments?user.id=0e95d295337b4472be946d252ac3bd7e&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:09.991737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aa4e7-f68c-4b78-b090-89855b4838ba None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:10.023122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e867ff-aac0-4ee3-a404-cab1bedb655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:10.023763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e867ff-aac0-4ee3-a404-cab1bedb655c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:10.023954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e867ff-aac0-4ee3-a404-cab1bedb655c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:10.024217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e867ff-aac0-4ee3-a404-cab1bedb655c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:10.049244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e867ff-aac0-4ee3-a404-cab1bedb655c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:10.050861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e867ff-aac0-4ee3-a404-cab1bedb655c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b7e867ff-aac0-4ee3-a404-cab1bedb655c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:10.081831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:10.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:10.083629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:10.083629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:10.083629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:10.083629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 268/535] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:10.093546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e867ff-aac0-4ee3-a404-cab1bedb655c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:10.119553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aa4e7-f68c-4b78-b090-89855b4838ba None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:10.120690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:10.161153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936df23-2f52-4109-8cb7-b22cbb180920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:10.162147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936df23-2f52-4109-8cb7-b22cbb180920 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:10.162624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936df23-2f52-4109-8cb7-b22cbb180920 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:10.163111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936df23-2f52-4109-8cb7-b22cbb180920 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:10.164001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936df23-2f52-4109-8cb7-b22cbb180920 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:10.164516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936df23-2f52-4109-8cb7-b22cbb180920 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:10.166146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936df23-2f52-4109-8cb7-b22cbb180920 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:10.187636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 268/536] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:10.193775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936df23-2f52-4109-8cb7-b22cbb180920 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:10.205511 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:10.315421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936df23-2f52-4109-8cb7-b22cbb180920 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:10.322661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e867ff-aac0-4ee3-a404-cab1bedb655c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:10.356659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93904e27-954e-4490-8b48-c334228a7d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:10.358190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93904e27-954e-4490-8b48-c334228a7d17 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:10.358730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93904e27-954e-4490-8b48-c334228a7d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:10.359411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93904e27-954e-4490-8b48-c334228a7d17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:10.360340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93904e27-954e-4490-8b48-c334228a7d17 None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=5b4aef937fb047f6869a156a68e7bf2f) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:10.361362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93904e27-954e-4490-8b48-c334228a7d17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:10.363025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93904e27-954e-4490-8b48-c334228a7d17 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:10.369677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 269/537] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:10 2022] GET /identity/v3/role_assignments?user.id=5b4aef937fb047f6869a156a68e7bf2f&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:10.391615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f03701-d786-4221-b22c-d426f4af0f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:10.392237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f03701-d786-4221-b22c-d426f4af0f09 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:10.392527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f03701-d786-4221-b22c-d426f4af0f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:10.392818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f03701-d786-4221-b22c-d426f4af0f09 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:10.393491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f03701-d786-4221-b22c-d426f4af0f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:10.393790 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f03701-d786-4221-b22c-d426f4af0f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:10.395246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f03701-d786-4221-b22c-d426f4af0f09 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:10.410745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 269/538] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:10.416266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93904e27-954e-4490-8b48-c334228a7d17 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:10.539061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:10.571967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93904e27-954e-4490-8b48-c334228a7d17 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:10.582053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fcdea-ac73-4798-ad09-e85ff0264c38 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:10.582333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fcdea-ac73-4798-ad09-e85ff0264c38 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:10.582606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fcdea-ac73-4798-ad09-e85ff0264c38 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:10.584342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 270/539] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:10.592479 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03921db-b511-4bc9-a251-0c5c567c4fb3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:10.592785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03921db-b511-4bc9-a251-0c5c567c4fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:10.593053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03921db-b511-4bc9-a251-0c5c567c4fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:10.617818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e03921db-b511-4bc9-a251-0c5c567c4fb3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:10.619315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e03921db-b511-4bc9-a251-0c5c567c4fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:10.649215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcb265-c248-4aec-808b-32ab0cebf113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:10.649860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcb265-c248-4aec-808b-32ab0cebf113 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:10.650215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcb265-c248-4aec-808b-32ab0cebf113 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:10.650418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcb265-c248-4aec-808b-32ab0cebf113 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:10.669270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcb265-c248-4aec-808b-32ab0cebf113 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:10.670757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcb265-c248-4aec-808b-32ab0cebf113 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8abcb265-c248-4aec-808b-32ab0cebf113 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:10.693819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:10.694748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:10.695585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:10.695585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:10.695585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:10.695585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:10.695585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:10.695585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:10.695585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:10.695585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:10.695585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 270/540] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:10.699709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 271/541] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:10.705417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcb265-c248-4aec-808b-32ab0cebf113 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:10.726632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03921db-b511-4bc9-a251-0c5c567c4fb3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:10.730799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:10.880598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcb265-c248-4aec-808b-32ab0cebf113 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:10.917094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa80758-9ffc-4aff-a894-1bd86a2eef78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:10.917701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa80758-9ffc-4aff-a894-1bd86a2eef78 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:10.917981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa80758-9ffc-4aff-a894-1bd86a2eef78 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:10.918254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa80758-9ffc-4aff-a894-1bd86a2eef78 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:10.918950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa80758-9ffc-4aff-a894-1bd86a2eef78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:10.919317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa80758-9ffc-4aff-a894-1bd86a2eef78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:10.921331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa80758-9ffc-4aff-a894-1bd86a2eef78 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:10.928484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:10.931340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03921db-b511-4bc9-a251-0c5c567c4fb3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:10.933950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 271/542] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:10.940319 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa80758-9ffc-4aff-a894-1bd86a2eef78 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:11.011333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a6669-5a78-45a4-a10f-64260515e2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:11.011974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a6669-5a78-45a4-a10f-64260515e2cc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:11.012249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a6669-5a78-45a4-a10f-64260515e2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:11.012530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a6669-5a78-45a4-a10f-64260515e2cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:11.053454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a6669-5a78-45a4-a10f-64260515e2cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4e2a6669-5a78-45a4-a10f-64260515e2cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:11.058196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:11.059099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:11.059966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 272/543] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:11.069410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a6669-5a78-45a4-a10f-64260515e2cc None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:11.076119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa80758-9ffc-4aff-a894-1bd86a2eef78 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:11.111757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13dcfa-9763-4541-b5ce-47a3afafe524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:11.112354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13dcfa-9763-4541-b5ce-47a3afafe524 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:11.112677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13dcfa-9763-4541-b5ce-47a3afafe524 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:11.113053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13dcfa-9763-4541-b5ce-47a3afafe524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:11.113784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13dcfa-9763-4541-b5ce-47a3afafe524 None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=95f02466042946ee96e37c5adc27b01d) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:11.114087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13dcfa-9763-4541-b5ce-47a3afafe524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:11.115526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13dcfa-9763-4541-b5ce-47a3afafe524 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:11.121817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 272/544] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:10 2022] GET /identity/v3/role_assignments?user.id=95f02466042946ee96e37c5adc27b01d&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:11.177979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:11.227614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a6669-5a78-45a4-a10f-64260515e2cc None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:11.275192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60292bd-9991-42a1-a1e7-07c8d4556bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:11.275799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60292bd-9991-42a1-a1e7-07c8d4556bca None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:11.276075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60292bd-9991-42a1-a1e7-07c8d4556bca None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:11.276342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60292bd-9991-42a1-a1e7-07c8d4556bca None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:11.277031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60292bd-9991-42a1-a1e7-07c8d4556bca None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:11.277326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60292bd-9991-42a1-a1e7-07c8d4556bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:11.279247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60292bd-9991-42a1-a1e7-07c8d4556bca None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:11.295196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 273/545] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:11.301284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13dcfa-9763-4541-b5ce-47a3afafe524 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:11.371516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:11.419365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13dcfa-9763-4541-b5ce-47a3afafe524 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:11.457080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22840f7-e296-4801-8007-54f65942127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:11.457702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22840f7-e296-4801-8007-54f65942127e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:11.457980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22840f7-e296-4801-8007-54f65942127e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:11.458248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22840f7-e296-4801-8007-54f65942127e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:11.470116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22840f7-e296-4801-8007-54f65942127e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:11.471586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22840f7-e296-4801-8007-54f65942127e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a22840f7-e296-4801-8007-54f65942127e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:11.487333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:11.488264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:11.489148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:11.489148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:11.489148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:11.489148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 273/546] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:11.498096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60292bd-9991-42a1-a1e7-07c8d4556bca None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:11.513890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c9961-3138-47f0-88a9-74dc07aa5d6b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:11.514170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c9961-3138-47f0-88a9-74dc07aa5d6b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:11.514702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c9961-3138-47f0-88a9-74dc07aa5d6b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:11.515858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 274/547] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:11.524796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22741-dde7-40a5-bbfb-09678d8048df None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:11.524796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22741-dde7-40a5-bbfb-09678d8048df None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:11.524796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22741-dde7-40a5-bbfb-09678d8048df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:11.540743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7df22741-dde7-40a5-bbfb-09678d8048df None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:11.541922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7df22741-dde7-40a5-bbfb-09678d8048df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:11.631416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60292bd-9991-42a1-a1e7-07c8d4556bca None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:11.656852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 275/548] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:51:11.676921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:11.685360 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475c2ee-1d21-4403-9df4-fe27531d9c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:11.686221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475c2ee-1d21-4403-9df4-fe27531d9c89 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:11.686749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475c2ee-1d21-4403-9df4-fe27531d9c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:11.687243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475c2ee-1d21-4403-9df4-fe27531d9c89 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:11.688086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475c2ee-1d21-4403-9df4-fe27531d9c89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:11.689025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475c2ee-1d21-4403-9df4-fe27531d9c89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:11.690678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475c2ee-1d21-4403-9df4-fe27531d9c89 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:11.703643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df22741-dde7-40a5-bbfb-09678d8048df None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:11.705976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 274/549] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:11.712659 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475c2ee-1d21-4403-9df4-fe27531d9c89 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:11.760027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:11.843541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475c2ee-1d21-4403-9df4-fe27531d9c89 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:11.884998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f7035-fab9-447b-b063-5b3ef2f771b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:11.885627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f7035-fab9-447b-b063-5b3ef2f771b0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:11.885898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f7035-fab9-447b-b063-5b3ef2f771b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:11.886164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f7035-fab9-447b-b063-5b3ef2f771b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:11.894142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df22741-dde7-40a5-bbfb-09678d8048df None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:11.905252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f7035-fab9-447b-b063-5b3ef2f771b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:11.906716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f7035-fab9-447b-b063-5b3ef2f771b0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d45f7035-fab9-447b-b063-5b3ef2f771b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:11.924087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:11.925206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:11.926114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:11.929904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 275/550] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:11.931190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f7035-fab9-447b-b063-5b3ef2f771b0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:11.969257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ece86a-3b30-4fe9-9696-17b3687a8fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:11.970199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ece86a-3b30-4fe9-9696-17b3687a8fcb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:11.970665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ece86a-3b30-4fe9-9696-17b3687a8fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:11.971118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ece86a-3b30-4fe9-9696-17b3687a8fcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:11.981385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:11.995565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ece86a-3b30-4fe9-9696-17b3687a8fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-59ece86a-3b30-4fe9-9696-17b3687a8fcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:12.000981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:12.002067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:12.003496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 276/551] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:12.008899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ece86a-3b30-4fe9-9696-17b3687a8fcb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:12.042328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f7035-fab9-447b-b063-5b3ef2f771b0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:12.086794 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9edc9-f0ef-49dd-98a3-93a80bfd605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:12.088231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9edc9-f0ef-49dd-98a3-93a80bfd605f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.090260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9edc9-f0ef-49dd-98a3-93a80bfd605f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.090760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9edc9-f0ef-49dd-98a3-93a80bfd605f None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.092223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9edc9-f0ef-49dd-98a3-93a80bfd605f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:12.093093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9edc9-f0ef-49dd-98a3-93a80bfd605f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:12.097493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9edc9-f0ef-49dd-98a3-93a80bfd605f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:12.135258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 276/552] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 17:51:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:12.141033 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9edc9-f0ef-49dd-98a3-93a80bfd605f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:12.212619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:12.230733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ece86a-3b30-4fe9-9696-17b3687a8fcb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:12.284910 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c18463-a077-4ebc-b3f4-f79d4c937e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:12.285995 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c18463-a077-4ebc-b3f4-f79d4c937e27 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.286538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c18463-a077-4ebc-b3f4-f79d4c937e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.287008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c18463-a077-4ebc-b3f4-f79d4c937e27 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.287986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c18463-a077-4ebc-b3f4-f79d4c937e27 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:12.288485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c18463-a077-4ebc-b3f4-f79d4c937e27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:12.290214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c18463-a077-4ebc-b3f4-f79d4c937e27 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:12.300681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 277/553] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:12.302158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe23c9-5197-4c31-be51-79e5c07b9e57 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.302709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe23c9-5197-4c31-be51-79e5c07b9e57 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.303276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe23c9-5197-4c31-be51-79e5c07b9e57 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.308841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9edc9-f0ef-49dd-98a3-93a80bfd605f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:12.309613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:12.310573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 278/554] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:12.311645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe23c9-5197-4c31-be51-79e5c07b9e57 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:12.350143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266a8ea-1ea8-4e32-b1c5-3dd5d3fa37a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:12.350747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266a8ea-1ea8-4e32-b1c5-3dd5d3fa37a9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.351019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266a8ea-1ea8-4e32-b1c5-3dd5d3fa37a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.351283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266a8ea-1ea8-4e32-b1c5-3dd5d3fa37a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.427390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe23c9-5197-4c31-be51-79e5c07b9e57 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:12.443212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266a8ea-1ea8-4e32-b1c5-3dd5d3fa37a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:12.445000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266a8ea-1ea8-4e32-b1c5-3dd5d3fa37a9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:12.471627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8906c-177c-4364-8b1b-383700a4dcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:12.472256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8906c-177c-4364-8b1b-383700a4dcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.472549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8906c-177c-4364-8b1b-383700a4dcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.472818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8906c-177c-4364-8b1b-383700a4dcd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.488762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8906c-177c-4364-8b1b-383700a4dcd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:12.490245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8906c-177c-4364-8b1b-383700a4dcd9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-bef8906c-177c-4364-8b1b-383700a4dcd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:12.509327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:12.510304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:12.511190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:12.511190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:12.511190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:12.511190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 279/555] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:12.512024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7c21b-c1e8-4824-ae5f-c6e4ec1ef1cf None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.512300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7c21b-c1e8-4824-ae5f-c6e4ec1ef1cf None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.512640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7c21b-c1e8-4824-ae5f-c6e4ec1ef1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.541836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0ce7c21b-c1e8-4824-ae5f-c6e4ec1ef1cf None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:12.543276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0ce7c21b-c1e8-4824-ae5f-c6e4ec1ef1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:12.547266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:12.565597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 277/556] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:12.567200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266a8ea-1ea8-4e32-b1c5-3dd5d3fa37a9 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:12.572105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:12.633644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 280/557] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:12.633644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7c21b-c1e8-4824-ae5f-c6e4ec1ef1cf None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:12.700198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266a8ea-1ea8-4e32-b1c5-3dd5d3fa37a9 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:12.747893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65db575-f237-4327-8d52-40043c507eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:12.748409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65db575-f237-4327-8d52-40043c507eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.748713 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65db575-f237-4327-8d52-40043c507eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.748979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65db575-f237-4327-8d52-40043c507eb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.749610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65db575-f237-4327-8d52-40043c507eb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:12.749902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65db575-f237-4327-8d52-40043c507eb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:12.750899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7c21b-c1e8-4824-ae5f-c6e4ec1ef1cf None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:12.751273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65db575-f237-4327-8d52-40043c507eb5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:12.764619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 278/558] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:12.765459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65db575-f237-4327-8d52-40043c507eb5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:12.792345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33294e-ffa0-4e39-af7e-b07ad9cada54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:12.793269 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33294e-ffa0-4e39-af7e-b07ad9cada54 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.793699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33294e-ffa0-4e39-af7e-b07ad9cada54 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.794110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33294e-ffa0-4e39-af7e-b07ad9cada54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.804138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:12.805638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33294e-ffa0-4e39-af7e-b07ad9cada54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:12.806849 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33294e-ffa0-4e39-af7e-b07ad9cada54 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1f33294e-ffa0-4e39-af7e-b07ad9cada54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:12.821049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:12.822466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:12.823376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:12.823376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:12.823376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:12.826319 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 281/559] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:12.827544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010acdb8-8c7b-4e04-b9b3-f8eff20dfe42 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.827824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010acdb8-8c7b-4e04-b9b3-f8eff20dfe42 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.828094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010acdb8-8c7b-4e04-b9b3-f8eff20dfe42 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.829799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 282/560] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:12.834421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010acdb8-8c7b-4e04-b9b3-f8eff20dfe42 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:12.843253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:12.882911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65db575-f237-4327-8d52-40043c507eb5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:12.917275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f69f5-cd54-479a-b52a-807c93f7f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:12.917890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f69f5-cd54-479a-b52a-807c93f7f316 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.918169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f69f5-cd54-479a-b52a-807c93f7f316 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.918434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f69f5-cd54-479a-b52a-807c93f7f316 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.928601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f69f5-cd54-479a-b52a-807c93f7f316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-ee6f69f5-cd54-479a-b52a-807c93f7f316 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:12.934386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:12.935925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:12.936825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:12.940835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 279/561] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:12.941486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f69f5-cd54-479a-b52a-807c93f7f316 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:12.944321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010acdb8-8c7b-4e04-b9b3-f8eff20dfe42 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:12.979493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b53824-5fdf-4d8f-bf99-0c729e10dc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:12.980111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b53824-5fdf-4d8f-bf99-0c729e10dc12 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:12.980385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b53824-5fdf-4d8f-bf99-0c729e10dc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:12.980683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b53824-5fdf-4d8f-bf99-0c729e10dc12 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:12.989618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b53824-5fdf-4d8f-bf99-0c729e10dc12 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:12.991067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b53824-5fdf-4d8f-bf99-0c729e10dc12 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-91b53824-5fdf-4d8f-bf99-0c729e10dc12 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:13.003258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:13.004267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 283/562] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 17:51:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f3931-1721-4f6b-9ace-2af6ae5eb9b7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.005882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f3931-1721-4f6b-9ace-2af6ae5eb9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.006640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f3931-1721-4f6b-9ace-2af6ae5eb9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.032431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7c8f3931-1721-4f6b-9ace-2af6ae5eb9b7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:13.033662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7c8f3931-1721-4f6b-9ace-2af6ae5eb9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:13.047903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f69f5-cd54-479a-b52a-807c93f7f316 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.072732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:13.081939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:13.085135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0c3a0-d91a-434d-9aa0-73caa97d509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.085735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0c3a0-d91a-434d-9aa0-73caa97d509d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.086010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0c3a0-d91a-434d-9aa0-73caa97d509d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.086284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0c3a0-d91a-434d-9aa0-73caa97d509d None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.086919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0c3a0-d91a-434d-9aa0-73caa97d509d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:13.087219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0c3a0-d91a-434d-9aa0-73caa97d509d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.088668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0c3a0-d91a-434d-9aa0-73caa97d509d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.104913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 284/563] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:13.104913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f3931-1721-4f6b-9ace-2af6ae5eb9b7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.112808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 280/564] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:13.113577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0c3a0-d91a-434d-9aa0-73caa97d509d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.216197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f3931-1721-4f6b-9ace-2af6ae5eb9b7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.220022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0c3a0-d91a-434d-9aa0-73caa97d509d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.256151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0833f-5fa1-4b67-bd48-4db8acd475e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.256706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0833f-5fa1-4b67-bd48-4db8acd475e6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.256985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0833f-5fa1-4b67-bd48-4db8acd475e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.257151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3309e3-31eb-416b-b3ea-8dfbbefcb6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.257261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0833f-5fa1-4b67-bd48-4db8acd475e6 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.257713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3309e3-31eb-416b-b3ea-8dfbbefcb6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.257955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0833f-5fa1-4b67-bd48-4db8acd475e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:13.258051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3309e3-31eb-416b-b3ea-8dfbbefcb6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.258241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3309e3-31eb-416b-b3ea-8dfbbefcb6c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.258346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0833f-5fa1-4b67-bd48-4db8acd475e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.258895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3309e3-31eb-416b-b3ea-8dfbbefcb6c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:13.259192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3309e3-31eb-416b-b3ea-8dfbbefcb6c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.259661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0833f-5fa1-4b67-bd48-4db8acd475e6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.260557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3309e3-31eb-416b-b3ea-8dfbbefcb6c0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.269194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 285/565] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:13.274179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3309e3-31eb-416b-b3ea-8dfbbefcb6c0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.278303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 281/566] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:13.278955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0833f-5fa1-4b67-bd48-4db8acd475e6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.324866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:13.326086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:13.383062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3309e3-31eb-416b-b3ea-8dfbbefcb6c0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.387342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0833f-5fa1-4b67-bd48-4db8acd475e6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.417097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6af19-1e00-4298-af67-03bd01b18a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.417726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6af19-1e00-4298-af67-03bd01b18a37 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.418000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6af19-1e00-4298-af67-03bd01b18a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.418267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6af19-1e00-4298-af67-03bd01b18a37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.425567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6af19-1e00-4298-af67-03bd01b18a37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.427004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6af19-1e00-4298-af67-03bd01b18a37 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.427246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a2f29-ce06-4bf8-9e88-a03c76647649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.427860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a2f29-ce06-4bf8-9e88-a03c76647649 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.428132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a2f29-ce06-4bf8-9e88-a03c76647649 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.428396 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a2f29-ce06-4bf8-9e88-a03c76647649 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.439276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a2f29-ce06-4bf8-9e88-a03c76647649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a3a6af19-1e00-4298-af67-03bd01b18a37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:13.439623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:13.441127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 286/567] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:13.442044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6af19-1e00-4298-af67-03bd01b18a37 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-611a2f29-ce06-4bf8-9e88-a03c76647649 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:13.448749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:13.449699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:13.450590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 282/568] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:13.451913 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a2f29-ce06-4bf8-9e88-a03c76647649 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.555286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6af19-1e00-4298-af67-03bd01b18a37 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.557736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a2f29-ce06-4bf8-9e88-a03c76647649 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.559452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:13.561231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:13.594377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d406a-692c-4a4f-9f44-ad5df151855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.594997 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d406a-692c-4a4f-9f44-ad5df151855b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.595280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d406a-692c-4a4f-9f44-ad5df151855b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.595614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d406a-692c-4a4f-9f44-ad5df151855b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.597309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3fe7e-274b-4502-9ac9-66eff360cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.597951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3fe7e-274b-4502-9ac9-66eff360cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.598237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3fe7e-274b-4502-9ac9-66eff360cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.598500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3fe7e-274b-4502-9ac9-66eff360cdd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.606017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d406a-692c-4a4f-9f44-ad5df151855b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.612737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d406a-692c-4a4f-9f44-ad5df151855b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.612846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3fe7e-274b-4502-9ac9-66eff360cdd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.612846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3fe7e-274b-4502-9ac9-66eff360cdd4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-cd9d406a-692c-4a4f-9f44-ad5df151855b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:13.622501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:13.623436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:13.624323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:13.624323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:13.624323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:13.624323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 283/569] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-29b3fe7e-274b-4502-9ac9-66eff360cdd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:13.624988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:13.626136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:13.627280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:13.627280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:13.627280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:13.627449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d406a-692c-4a4f-9f44-ad5df151855b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.627598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 287/570] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:13.627598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3fe7e-274b-4502-9ac9-66eff360cdd4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.724751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d406a-692c-4a4f-9f44-ad5df151855b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.724961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3fe7e-274b-4502-9ac9-66eff360cdd4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.757148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587077b8-5f73-4044-99ef-1baf72203fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.757509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57b27b-e030-4582-a617-5a00927bab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.757775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587077b8-5f73-4044-99ef-1baf72203fff None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.758053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587077b8-5f73-4044-99ef-1baf72203fff None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.758157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57b27b-e030-4582-a617-5a00927bab64 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.758325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587077b8-5f73-4044-99ef-1baf72203fff None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.758429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57b27b-e030-4582-a617-5a00927bab64 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.758589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57b27b-e030-4582-a617-5a00927bab64 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.759010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587077b8-5f73-4044-99ef-1baf72203fff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:13.759232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57b27b-e030-4582-a617-5a00927bab64 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:13.759341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587077b8-5f73-4044-99ef-1baf72203fff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.759521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57b27b-e030-4582-a617-5a00927bab64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.760755 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587077b8-5f73-4044-99ef-1baf72203fff None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.760889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57b27b-e030-4582-a617-5a00927bab64 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.772425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 288/571] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:13.772425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098ece9-7222-428f-b413-eb88dc09c3ad None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.772425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098ece9-7222-428f-b413-eb88dc09c3ad None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.772425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098ece9-7222-428f-b413-eb88dc09c3ad None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.772425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 289/572] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:13.772857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098ece9-7222-428f-b413-eb88dc09c3ad None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.776241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 284/573] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:13.777090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587077b8-5f73-4044-99ef-1baf72203fff None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.795906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:13.796459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:13.877378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098ece9-7222-428f-b413-eb88dc09c3ad None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.882614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587077b8-5f73-4044-99ef-1baf72203fff None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:13.910467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab782a9-19b5-4da1-a914-c42de887bea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.911044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab782a9-19b5-4da1-a914-c42de887bea7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.911318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab782a9-19b5-4da1-a914-c42de887bea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.911574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab782a9-19b5-4da1-a914-c42de887bea7 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.912195 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab782a9-19b5-4da1-a914-c42de887bea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:13.912485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab782a9-19b5-4da1-a914-c42de887bea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.913856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab782a9-19b5-4da1-a914-c42de887bea7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.915839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bd1c7-2efb-442b-8a8f-aee654b6de96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:13.916430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bd1c7-2efb-442b-8a8f-aee654b6de96 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.916734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bd1c7-2efb-442b-8a8f-aee654b6de96 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.917002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bd1c7-2efb-442b-8a8f-aee654b6de96 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.917629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bd1c7-2efb-442b-8a8f-aee654b6de96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:13.917925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bd1c7-2efb-442b-8a8f-aee654b6de96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:13.919816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bd1c7-2efb-442b-8a8f-aee654b6de96 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:13.925568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 290/574] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:13.926331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab782a9-19b5-4da1-a914-c42de887bea7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:13.935229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 285/575] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:13.936727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3823eaa-6930-45ca-a733-aeafba553bfe None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:13.937008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3823eaa-6930-45ca-a733-aeafba553bfe None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:13.937278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3823eaa-6930-45ca-a733-aeafba553bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:13.955034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e3823eaa-6930-45ca-a733-aeafba553bfe None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:13.956203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e3823eaa-6930-45ca-a733-aeafba553bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:14.008090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:14.020896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 286/576] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:14.021624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3823eaa-6930-45ca-a733-aeafba553bfe None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.024770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:14.030291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab782a9-19b5-4da1-a914-c42de887bea7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.064512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dca185-248b-41b7-af8c-8b33bf06098a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.065142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dca185-248b-41b7-af8c-8b33bf06098a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.065432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dca185-248b-41b7-af8c-8b33bf06098a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.065696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dca185-248b-41b7-af8c-8b33bf06098a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.075210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dca185-248b-41b7-af8c-8b33bf06098a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.077160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dca185-248b-41b7-af8c-8b33bf06098a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-01dca185-248b-41b7-af8c-8b33bf06098a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:14.091762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:14.092760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:14.093653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:14.093653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:14.093653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:14.096644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 291/577] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:14.097638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dca185-248b-41b7-af8c-8b33bf06098a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.119623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3823eaa-6930-45ca-a733-aeafba553bfe None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.152598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d59a58-603a-4349-a79e-8afb1f83dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.153126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d59a58-603a-4349-a79e-8afb1f83dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.153395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d59a58-603a-4349-a79e-8afb1f83dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.154219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d59a58-603a-4349-a79e-8afb1f83dbc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.154920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d59a58-603a-4349-a79e-8afb1f83dbc1 None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=3375c3134ebe473cb5da29f083321cdb) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:14.155216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d59a58-603a-4349-a79e-8afb1f83dbc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.156662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d59a58-603a-4349-a79e-8afb1f83dbc1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.161823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 287/578] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/role_assignments?user.id=3375c3134ebe473cb5da29f083321cdb&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:14.162638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d59a58-603a-4349-a79e-8afb1f83dbc1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.197979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dca185-248b-41b7-af8c-8b33bf06098a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.229556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:14.235976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6538acf-2cd0-4f7d-9e54-d6308d0be63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.236634 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6538acf-2cd0-4f7d-9e54-d6308d0be63d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.236919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6538acf-2cd0-4f7d-9e54-d6308d0be63d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.237166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6538acf-2cd0-4f7d-9e54-d6308d0be63d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.239235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:14.244461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6538acf-2cd0-4f7d-9e54-d6308d0be63d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.246015 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6538acf-2cd0-4f7d-9e54-d6308d0be63d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f6538acf-2cd0-4f7d-9e54-d6308d0be63d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:14.258041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:14.259221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:14.260315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:14.260315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:14.260315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:14.260315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:14.260315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:14.260315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:14.260315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:14.260315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:14.264982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 292/579] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:14.266853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6538acf-2cd0-4f7d-9e54-d6308d0be63d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.280780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d59a58-603a-4349-a79e-8afb1f83dbc1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.319081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057c3b4-addd-4612-861e-9f04ccb9ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.320603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057c3b4-addd-4612-861e-9f04ccb9ac50 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.320947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057c3b4-addd-4612-861e-9f04ccb9ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.321281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057c3b4-addd-4612-861e-9f04ccb9ac50 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.329827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057c3b4-addd-4612-861e-9f04ccb9ac50 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.331300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057c3b4-addd-4612-861e-9f04ccb9ac50 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9057c3b4-addd-4612-861e-9f04ccb9ac50 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:14.344025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:14.345245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:14.346218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 288/580] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:14.346807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057c3b4-addd-4612-861e-9f04ccb9ac50 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.386862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6538acf-2cd0-4f7d-9e54-d6308d0be63d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.422805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aba4aa-5b0f-402c-b4f6-9a6bf92e1bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.423352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aba4aa-5b0f-402c-b4f6-9a6bf92e1bed None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.423625 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aba4aa-5b0f-402c-b4f6-9a6bf92e1bed None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.423889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aba4aa-5b0f-402c-b4f6-9a6bf92e1bed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.435334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aba4aa-5b0f-402c-b4f6-9a6bf92e1bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-e9aba4aa-5b0f-402c-b4f6-9a6bf92e1bed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:14.441236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:14.442621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 293/581] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:14.443481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aba4aa-5b0f-402c-b4f6-9a6bf92e1bed None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.464527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057c3b4-addd-4612-861e-9f04ccb9ac50 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.494486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:14.499151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:14.504218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9aa2f-a824-48fd-8711-9ae8b8df0c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.504841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9aa2f-a824-48fd-8711-9ae8b8df0c17 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.505115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9aa2f-a824-48fd-8711-9ae8b8df0c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.513857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9aa2f-a824-48fd-8711-9ae8b8df0c17 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.513857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9aa2f-a824-48fd-8711-9ae8b8df0c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:14.513857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9aa2f-a824-48fd-8711-9ae8b8df0c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.513857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9aa2f-a824-48fd-8711-9ae8b8df0c17 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.520903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 289/582] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:14.521828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9aa2f-a824-48fd-8711-9ae8b8df0c17 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.549722 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aba4aa-5b0f-402c-b4f6-9a6bf92e1bed None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.584408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29fcd2-6ef4-4826-8c96-0d25add89198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.585106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29fcd2-6ef4-4826-8c96-0d25add89198 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.585392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29fcd2-6ef4-4826-8c96-0d25add89198 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.585659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29fcd2-6ef4-4826-8c96-0d25add89198 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.586355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29fcd2-6ef4-4826-8c96-0d25add89198 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:14.586631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29fcd2-6ef4-4826-8c96-0d25add89198 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.588074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29fcd2-6ef4-4826-8c96-0d25add89198 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.602134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 294/583] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:14.602983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29fcd2-6ef4-4826-8c96-0d25add89198 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.640594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9aa2f-a824-48fd-8711-9ae8b8df0c17 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.677835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f14de-bff2-4e36-84a2-50bc67888a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.678357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f14de-bff2-4e36-84a2-50bc67888a1d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.678626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f14de-bff2-4e36-84a2-50bc67888a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.678888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f14de-bff2-4e36-84a2-50bc67888a1d None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.679603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f14de-bff2-4e36-84a2-50bc67888a1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:14.679898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f14de-bff2-4e36-84a2-50bc67888a1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.681331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f14de-bff2-4e36-84a2-50bc67888a1d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.698544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 290/584] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:14.699346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f14de-bff2-4e36-84a2-50bc67888a1d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.711992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29fcd2-6ef4-4826-8c96-0d25add89198 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.723929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:14.748623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:14.751716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52c32b-d8f9-40ec-84ac-7efcb65e5c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.752317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52c32b-d8f9-40ec-84ac-7efcb65e5c5c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.752630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52c32b-d8f9-40ec-84ac-7efcb65e5c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.752899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52c32b-d8f9-40ec-84ac-7efcb65e5c5c None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.753553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52c32b-d8f9-40ec-84ac-7efcb65e5c5c None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:14.753852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52c32b-d8f9-40ec-84ac-7efcb65e5c5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.755205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52c32b-d8f9-40ec-84ac-7efcb65e5c5c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.769368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 295/585] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:14.769368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52c32b-d8f9-40ec-84ac-7efcb65e5c5c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.818658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f14de-bff2-4e36-84a2-50bc67888a1d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.855574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ef670-62ab-416c-87aa-8d2f6e6abd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.856198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ef670-62ab-416c-87aa-8d2f6e6abd30 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.856462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ef670-62ab-416c-87aa-8d2f6e6abd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.856781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ef670-62ab-416c-87aa-8d2f6e6abd30 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.864078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ef670-62ab-416c-87aa-8d2f6e6abd30 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.865538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ef670-62ab-416c-87aa-8d2f6e6abd30 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.875925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52c32b-d8f9-40ec-84ac-7efcb65e5c5c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-cd6ef670-62ab-416c-87aa-8d2f6e6abd30 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:14.878838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:14.879974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 17:51:14.881089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:14.881692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 291/586] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:14.883459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ef670-62ab-416c-87aa-8d2f6e6abd30 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:14.923730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ebce9-524b-421a-9dda-d2497105d261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:14.924385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ebce9-524b-421a-9dda-d2497105d261 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:14.924733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ebce9-524b-421a-9dda-d2497105d261 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:14.925002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ebce9-524b-421a-9dda-d2497105d261 None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/c496da229db34f90a4b3fdcfa582dca8/roles/2978a8a56cd84b4b852dc6738c5c54c3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:14.940313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ebce9-524b-421a-9dda-d2497105d261 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=c496da229db34f90a4b3fdcfa582dca8, role_id=2978a8a56cd84b4b852dc6738c5c54c3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:14.941739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ebce9-524b-421a-9dda-d2497105d261 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:14.987384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ef670-62ab-416c-87aa-8d2f6e6abd30 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:14.994567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:15.009378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 296/587] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:51:14 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/c496da229db34f90a4b3fdcfa582dca8/roles/2978a8a56cd84b4b852dc6738c5c54c3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:51:15.017756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ebce9-524b-421a-9dda-d2497105d261 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.021282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3147d-f144-4c47-ba42-48372ea0c266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.021926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3147d-f144-4c47-ba42-48372ea0c266 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.022198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3147d-f144-4c47-ba42-48372ea0c266 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.022455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3147d-f144-4c47-ba42-48372ea0c266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.039809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3147d-f144-4c47-ba42-48372ea0c266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.041905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3147d-f144-4c47-ba42-48372ea0c266 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.058990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-30b3147d-f144-4c47-ba42-48372ea0c266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:15.063614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:15.064707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:15.065578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:15.065578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:15.065578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:15.065578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 292/588] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:15.066459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3147d-f144-4c47-ba42-48372ea0c266 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.180464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ebce9-524b-421a-9dda-d2497105d261 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.206267 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3147d-f144-4c47-ba42-48372ea0c266 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.231981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d60d-c7d1-49a1-92f5-685331fc9af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.232663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d60d-c7d1-49a1-92f5-685331fc9af3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.232981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d60d-c7d1-49a1-92f5-685331fc9af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.233254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d60d-c7d1-49a1-92f5-685331fc9af3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.243627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d60d-c7d1-49a1-92f5-685331fc9af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.245112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d60d-c7d1-49a1-92f5-685331fc9af3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.246023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf8c59-203b-4f4b-9ff2-70bf1929a91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.246854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf8c59-203b-4f4b-9ff2-70bf1929a91f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.247943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf8c59-203b-4f4b-9ff2-70bf1929a91f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.248403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf8c59-203b-4f4b-9ff2-70bf1929a91f None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.249335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf8c59-203b-4f4b-9ff2-70bf1929a91f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:15.249844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf8c59-203b-4f4b-9ff2-70bf1929a91f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.251512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf8c59-203b-4f4b-9ff2-70bf1929a91f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4c70d60d-c7d1-49a1-92f5-685331fc9af3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:15.260393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:15.261546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:15.262451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:15.262451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:15.262451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:15.262451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 297/589] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:15.263751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d60d-c7d1-49a1-92f5-685331fc9af3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.276052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 293/590] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:15.277284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf8c59-203b-4f4b-9ff2-70bf1929a91f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.296482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:15.369009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:15.386398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d60d-c7d1-49a1-92f5-685331fc9af3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.398339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf8c59-203b-4f4b-9ff2-70bf1929a91f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.430099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ced88-c418-48c2-a476-486760d1a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.431226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ced88-c418-48c2-a476-486760d1a275 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.431629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ced88-c418-48c2-a476-486760d1a275 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.432205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ced88-c418-48c2-a476-486760d1a275 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.432968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ced88-c418-48c2-a476-486760d1a275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:15.433362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ced88-c418-48c2-a476-486760d1a275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.435493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ced88-c418-48c2-a476-486760d1a275 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.436785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd166e-dcf4-4dce-b9a7-0b0e5ad37f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.437549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd166e-dcf4-4dce-b9a7-0b0e5ad37f03 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.437965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd166e-dcf4-4dce-b9a7-0b0e5ad37f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.438371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd166e-dcf4-4dce-b9a7-0b0e5ad37f03 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.439121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd166e-dcf4-4dce-b9a7-0b0e5ad37f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:15.439562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd166e-dcf4-4dce-b9a7-0b0e5ad37f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.441131 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd166e-dcf4-4dce-b9a7-0b0e5ad37f03 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.451101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 298/591] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:15.452320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ced88-c418-48c2-a476-486760d1a275 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.454333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 294/592] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:15.459221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd166e-dcf4-4dce-b9a7-0b0e5ad37f03 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.542484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:15.569377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd166e-dcf4-4dce-b9a7-0b0e5ad37f03 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.579286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ced88-c418-48c2-a476-486760d1a275 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.609647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12914047-89e0-4c2b-94db-dd04b85dc209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.610632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12914047-89e0-4c2b-94db-dd04b85dc209 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.611141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12914047-89e0-4c2b-94db-dd04b85dc209 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12914047-89e0-4c2b-94db-dd04b85dc209 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.618964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12914047-89e0-4c2b-94db-dd04b85dc209 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.620761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12914047-89e0-4c2b-94db-dd04b85dc209 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.621926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc48ebf-d6dc-4410-868a-1b2cab336adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.623126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc48ebf-d6dc-4410-868a-1b2cab336adb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.623395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc48ebf-d6dc-4410-868a-1b2cab336adb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.623658 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc48ebf-d6dc-4410-868a-1b2cab336adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.626029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-12914047-89e0-4c2b-94db-dd04b85dc209 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:15.634416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:15.641935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 295/593] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:15.642836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12914047-89e0-4c2b-94db-dd04b85dc209 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.643374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc48ebf-d6dc-4410-868a-1b2cab336adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.643374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc48ebf-d6dc-4410-868a-1b2cab336adb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-5dc48ebf-d6dc-4410-868a-1b2cab336adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:15.659166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:15.660409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:15.661553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:15.661553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:15.661553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:15.663464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 299/594] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:15.664378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc48ebf-d6dc-4410-868a-1b2cab336adb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.760559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12914047-89e0-4c2b-94db-dd04b85dc209 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.778932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc48ebf-d6dc-4410-868a-1b2cab336adb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.805959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147ac7a-4fa3-4dae-8002-8712472780cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.806739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147ac7a-4fa3-4dae-8002-8712472780cb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.807273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147ac7a-4fa3-4dae-8002-8712472780cb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.807641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147ac7a-4fa3-4dae-8002-8712472780cb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.811627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:15.817205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151b24-85fe-4ab1-9034-d5b19daba867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.818004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151b24-85fe-4ab1-9034-d5b19daba867 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.818406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151b24-85fe-4ab1-9034-d5b19daba867 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.818533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147ac7a-4fa3-4dae-8002-8712472780cb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.818921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151b24-85fe-4ab1-9034-d5b19daba867 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.819762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01151b24-85fe-4ab1-9034-d5b19daba867 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:15.820197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01151b24-85fe-4ab1-9034-d5b19daba867 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.820325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147ac7a-4fa3-4dae-8002-8712472780cb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.822019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01151b24-85fe-4ab1-9034-d5b19daba867 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4147ac7a-4fa3-4dae-8002-8712472780cb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:15.832533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:15.833699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 17:51:15.834878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:15.836599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 296/595] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:15.837295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147ac7a-4fa3-4dae-8002-8712472780cb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.842952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 300/596] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:15.844056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151b24-85fe-4ab1-9034-d5b19daba867 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:15.893320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:15.944612 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147ac7a-4fa3-4dae-8002-8712472780cb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.948688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151b24-85fe-4ab1-9034-d5b19daba867 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:15.980258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64603ad9-fa1f-4b5c-a24c-64fe9af32e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.981031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64603ad9-fa1f-4b5c-a24c-64fe9af32e8a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.981406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64603ad9-fa1f-4b5c-a24c-64fe9af32e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.982418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64603ad9-fa1f-4b5c-a24c-64fe9af32e8a None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.983161 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64603ad9-fa1f-4b5c-a24c-64fe9af32e8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:15.983571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64603ad9-fa1f-4b5c-a24c-64fe9af32e8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.985287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64603ad9-fa1f-4b5c-a24c-64fe9af32e8a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:15.986031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0e595-444e-48f6-853c-0dafa639cd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:15.986812 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0e595-444e-48f6-853c-0dafa639cd8c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:15.987279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0e595-444e-48f6-853c-0dafa639cd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:15.987684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0e595-444e-48f6-853c-0dafa639cd8c None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:15.988561 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0e595-444e-48f6-853c-0dafa639cd8c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:15.989015 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0e595-444e-48f6-853c-0dafa639cd8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:15.990622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0e595-444e-48f6-853c-0dafa639cd8c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:16.002907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 297/597] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:16.003865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64603ad9-fa1f-4b5c-a24c-64fe9af32e8a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:16.009093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 301/598] 10.222.0.50 () {56 vars in 1139 bytes} [Mon Aug 8 17:51:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:16.010059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0e595-444e-48f6-853c-0dafa639cd8c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:16.063783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:16.109714 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64603ad9-fa1f-4b5c-a24c-64fe9af32e8a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:16.116218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0e595-444e-48f6-853c-0dafa639cd8c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:16.118851 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:16.146898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b32e2c-7df7-4147-b6a6-4b1c04e25e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:16.147776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b32e2c-7df7-4147-b6a6-4b1c04e25e22 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:16.148159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b32e2c-7df7-4147-b6a6-4b1c04e25e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:16.148552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b32e2c-7df7-4147-b6a6-4b1c04e25e22 None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/0e95d295337b4472be946d252ac3bd7e/roles/2978a8a56cd84b4b852dc6738c5c54c3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:16.153163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125274b5-15a0-4dc9-a1f7-9d2e5c431b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:16.154498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125274b5-15a0-4dc9-a1f7-9d2e5c431b38 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:16.154924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125274b5-15a0-4dc9-a1f7-9d2e5c431b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:16.155326 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125274b5-15a0-4dc9-a1f7-9d2e5c431b38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:16.163383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125274b5-15a0-4dc9-a1f7-9d2e5c431b38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:16.165058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125274b5-15a0-4dc9-a1f7-9d2e5c431b38 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:16.165643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b32e2c-7df7-4147-b6a6-4b1c04e25e22 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=0e95d295337b4472be946d252ac3bd7e, role_id=2978a8a56cd84b4b852dc6738c5c54c3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:16.167219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b32e2c-7df7-4147-b6a6-4b1c04e25e22 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-125274b5-15a0-4dc9-a1f7-9d2e5c431b38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:16.178332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:16.185291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 302/599] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:16.186207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125274b5-15a0-4dc9-a1f7-9d2e5c431b38 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:16.236193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 298/600] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:51:16 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/0e95d295337b4472be946d252ac3bd7e/roles/2978a8a56cd84b4b852dc6738c5c54c3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:51:16.241868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad127e4-c366-491a-80ed-30f23ccc8222 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:16.242144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad127e4-c366-491a-80ed-30f23ccc8222 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:16.242410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad127e4-c366-491a-80ed-30f23ccc8222 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:16.243814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 299/601] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:16.244387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad127e4-c366-491a-80ed-30f23ccc8222 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:16.298061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125274b5-15a0-4dc9-a1f7-9d2e5c431b38 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:16.344822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d9494-a2bc-4a00-be7b-12a9fad3d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:16.346010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d9494-a2bc-4a00-be7b-12a9fad3d988 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:16.346899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d9494-a2bc-4a00-be7b-12a9fad3d988 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:16.347598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d9494-a2bc-4a00-be7b-12a9fad3d988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:16.361981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d9494-a2bc-4a00-be7b-12a9fad3d988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:16.364724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d9494-a2bc-4a00-be7b-12a9fad3d988 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-2d8d9494-a2bc-4a00-be7b-12a9fad3d988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:16.386399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:16.387613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:16.389289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:16.389289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:16.389289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:16.391077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 303/602] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:16.391500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad127e4-c366-491a-80ed-30f23ccc8222 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:16.392779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a175b15-7ba0-4a90-a581-664e463c8d44 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:16.393151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a175b15-7ba0-4a90-a581-664e463c8d44 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:16.393501 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a175b15-7ba0-4a90-a581-664e463c8d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:16.400978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:16.415140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4a175b15-7ba0-4a90-a581-664e463c8d44 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:16.416966 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4a175b15-7ba0-4a90-a581-664e463c8d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:16.431006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:16.432801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c80637-5b33-49ba-9ff8-11fbfad33ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:16.433580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c80637-5b33-49ba-9ff8-11fbfad33ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:16.433853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c80637-5b33-49ba-9ff8-11fbfad33ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:16.434118 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c80637-5b33-49ba-9ff8-11fbfad33ad5 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:16.435347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c80637-5b33-49ba-9ff8-11fbfad33ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:16.435646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c80637-5b33-49ba-9ff8-11fbfad33ad5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:16.437101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c80637-5b33-49ba-9ff8-11fbfad33ad5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:16.449860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 300/603] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:16.450799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c80637-5b33-49ba-9ff8-11fbfad33ad5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:16.515617 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 304/604] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:16.516459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a175b15-7ba0-4a90-a581-664e463c8d44 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:16.578144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c80637-5b33-49ba-9ff8-11fbfad33ad5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:16.615626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc2f6c-7843-486c-9ba2-3e224b363fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:16.616222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc2f6c-7843-486c-9ba2-3e224b363fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:16.616496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc2f6c-7843-486c-9ba2-3e224b363fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:16.616812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc2f6c-7843-486c-9ba2-3e224b363fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:16.617452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc2f6c-7843-486c-9ba2-3e224b363fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:16.617748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc2f6c-7843-486c-9ba2-3e224b363fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:16.619163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc2f6c-7843-486c-9ba2-3e224b363fa1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:16.632518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 301/605] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:16.633371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc2f6c-7843-486c-9ba2-3e224b363fa1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:16.661736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:16.665467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a175b15-7ba0-4a90-a581-664e463c8d44 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:16.710220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e387a-8d8e-4fba-bbfe-324fed69e582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:16.711058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e387a-8d8e-4fba-bbfe-324fed69e582 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:16.711434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e387a-8d8e-4fba-bbfe-324fed69e582 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:16.711805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e387a-8d8e-4fba-bbfe-324fed69e582 None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/5b4aef937fb047f6869a156a68e7bf2f/roles/2978a8a56cd84b4b852dc6738c5c54c3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:16.732757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e387a-8d8e-4fba-bbfe-324fed69e582 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=5b4aef937fb047f6869a156a68e7bf2f, role_id=2978a8a56cd84b4b852dc6738c5c54c3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:16.734344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e387a-8d8e-4fba-bbfe-324fed69e582 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:16.737754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:16.746993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc2f6c-7843-486c-9ba2-3e224b363fa1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:16.788536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd395443-8f97-4485-a36b-1c06b750421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:16.788536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd395443-8f97-4485-a36b-1c06b750421d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:16.788536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd395443-8f97-4485-a36b-1c06b750421d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:16.788536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd395443-8f97-4485-a36b-1c06b750421d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:16.806926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd395443-8f97-4485-a36b-1c06b750421d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-cd395443-8f97-4485-a36b-1c06b750421d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:16.811712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:16.812889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:16.813980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:16.814777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 302/606] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:16.815059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd395443-8f97-4485-a36b-1c06b750421d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:16.834545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 305/607] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:51:16 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/5b4aef937fb047f6869a156a68e7bf2f/roles/2978a8a56cd84b4b852dc6738c5c54c3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:51:16.835992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e387a-8d8e-4fba-bbfe-324fed69e582 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:16.978907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:16.995995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd395443-8f97-4485-a36b-1c06b750421d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:17.038437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe251e6-ea48-4fa9-9d28-291d80e511a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:17.039416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe251e6-ea48-4fa9-9d28-291d80e511a4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.040197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe251e6-ea48-4fa9-9d28-291d80e511a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.041372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe251e6-ea48-4fa9-9d28-291d80e511a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.051054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe251e6-ea48-4fa9-9d28-291d80e511a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:17.053083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe251e6-ea48-4fa9-9d28-291d80e511a4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:17.056466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e387a-8d8e-4fba-bbfe-324fed69e582 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4fe251e6-ea48-4fa9-9d28-291d80e511a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:17.067402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:17.068370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:17.069293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:17.069293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:17.069293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:17.069293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:17.069293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:17.069293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:17.069293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:17.069293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:17.070585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 303/608] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:17.076137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe251e6-ea48-4fa9-9d28-291d80e511a4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:17.104271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5583e7-3b5c-40d2-b811-ddd776d39b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:17.106242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5583e7-3b5c-40d2-b811-ddd776d39b88 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.106539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5583e7-3b5c-40d2-b811-ddd776d39b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.106812 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5583e7-3b5c-40d2-b811-ddd776d39b88 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.107471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5583e7-3b5c-40d2-b811-ddd776d39b88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:17.107762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5583e7-3b5c-40d2-b811-ddd776d39b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:17.109162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5583e7-3b5c-40d2-b811-ddd776d39b88 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:17.120827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 306/609] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:17.127006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5583e7-3b5c-40d2-b811-ddd776d39b88 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:17.196165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:17.214546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe251e6-ea48-4fa9-9d28-291d80e511a4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:17.248610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5583e7-3b5c-40d2-b811-ddd776d39b88 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:17.254583 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a0784-11d2-4bc1-95d7-36cdfcf04738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:17.255176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a0784-11d2-4bc1-95d7-36cdfcf04738 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.255567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a0784-11d2-4bc1-95d7-36cdfcf04738 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.256296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a0784-11d2-4bc1-95d7-36cdfcf04738 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.257016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a0784-11d2-4bc1-95d7-36cdfcf04738 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:17.257313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a0784-11d2-4bc1-95d7-36cdfcf04738 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:17.259168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a0784-11d2-4bc1-95d7-36cdfcf04738 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:17.276860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 304/610] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:17.278172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8fce9-3179-4d9a-8de2-f38e0b187d22 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.279011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8fce9-3179-4d9a-8de2-f38e0b187d22 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.279276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8fce9-3179-4d9a-8de2-f38e0b187d22 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.280746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 305/611] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:17.284546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8fce9-3179-4d9a-8de2-f38e0b187d22 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:17.294615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:17.294799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74d48c-a837-4387-afbf-679abb9ffae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:17.295245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74d48c-a837-4387-afbf-679abb9ffae8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.295526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74d48c-a837-4387-afbf-679abb9ffae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.295790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74d48c-a837-4387-afbf-679abb9ffae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.307393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74d48c-a837-4387-afbf-679abb9ffae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:17.314246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74d48c-a837-4387-afbf-679abb9ffae8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ac74d48c-a837-4387-afbf-679abb9ffae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:17.323887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:17.324882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:17.326522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:17.326522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:17.326522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:17.326522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 307/612] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:17.328539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706124c-9818-4190-89c9-9baab89840b1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.328539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706124c-9818-4190-89c9-9baab89840b1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.328539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706124c-9818-4190-89c9-9baab89840b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.348481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7706124c-9818-4190-89c9-9baab89840b1 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:17.349777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7706124c-9818-4190-89c9-9baab89840b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:17.414785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8fce9-3179-4d9a-8de2-f38e0b187d22 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:17.445152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 308/613] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:17.446358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706124c-9818-4190-89c9-9baab89840b1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:17.471359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674e861-60b3-4cf9-a2de-256551b33a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:17.471359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674e861-60b3-4cf9-a2de-256551b33a85 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.471359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674e861-60b3-4cf9-a2de-256551b33a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.471359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674e861-60b3-4cf9-a2de-256551b33a85 None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/95f02466042946ee96e37c5adc27b01d/roles/2978a8a56cd84b4b852dc6738c5c54c3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.487504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674e861-60b3-4cf9-a2de-256551b33a85 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=95f02466042946ee96e37c5adc27b01d, role_id=2978a8a56cd84b4b852dc6738c5c54c3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:17.489725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674e861-60b3-4cf9-a2de-256551b33a85 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:17.514209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:17.568023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 306/614] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:51:17 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/95f02466042946ee96e37c5adc27b01d/roles/2978a8a56cd84b4b852dc6738c5c54c3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:51:17.573702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674e861-60b3-4cf9-a2de-256551b33a85 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:17.581322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706124c-9818-4190-89c9-9baab89840b1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:17.656399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:17.681284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cfba7-c5b1-4966-b3b8-887da913949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:17.681924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cfba7-c5b1-4966-b3b8-887da913949e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.682266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cfba7-c5b1-4966-b3b8-887da913949e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.682551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cfba7-c5b1-4966-b3b8-887da913949e None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.683338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cfba7-c5b1-4966-b3b8-887da913949e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:17.683545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cfba7-c5b1-4966-b3b8-887da913949e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:17.689649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cfba7-c5b1-4966-b3b8-887da913949e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:17.703119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 309/615] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:17.708132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cfba7-c5b1-4966-b3b8-887da913949e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:17.736644 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674e861-60b3-4cf9-a2de-256551b33a85 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:17.798822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b8266-4645-4e02-ab19-37c48be0ef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:17.799560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b8266-4645-4e02-ab19-37c48be0ef6d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.800429 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b8266-4645-4e02-ab19-37c48be0ef6d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.800746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b8266-4645-4e02-ab19-37c48be0ef6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.814901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b8266-4645-4e02-ab19-37c48be0ef6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-140b8266-4645-4e02-ab19-37c48be0ef6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:17.821584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:17.822704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:17.823722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 307/616] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:17.830536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b8266-4645-4e02-ab19-37c48be0ef6d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:17.844698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:17.848665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cfba7-c5b1-4966-b3b8-887da913949e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:17.902753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8026b-b6f3-4897-bade-f43be819d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:17.903942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8026b-b6f3-4897-bade-f43be819d97a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:17.904219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8026b-b6f3-4897-bade-f43be819d97a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:17.904488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8026b-b6f3-4897-bade-f43be819d97a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:17.919120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8026b-b6f3-4897-bade-f43be819d97a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:17.920629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8026b-b6f3-4897-bade-f43be819d97a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-12a8026b-b6f3-4897-bade-f43be819d97a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:17.950820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:17.951818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:17.952802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 310/617] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:17.968111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8026b-b6f3-4897-bade-f43be819d97a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:17.982329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b8266-4645-4e02-ab19-37c48be0ef6d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:18.004702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:18.018710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b13cb9-b5ee-4361-9017-959f465f16b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:18.019322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b13cb9-b5ee-4361-9017-959f465f16b1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.019665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b13cb9-b5ee-4361-9017-959f465f16b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.019940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b13cb9-b5ee-4361-9017-959f465f16b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.020812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b13cb9-b5ee-4361-9017-959f465f16b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:18.020937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b13cb9-b5ee-4361-9017-959f465f16b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:18.022297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b13cb9-b5ee-4361-9017-959f465f16b1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:18.031541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 308/618] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:18.037222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b13cb9-b5ee-4361-9017-959f465f16b1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:18.128676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8026b-b6f3-4897-bade-f43be819d97a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:18.157473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b13cb9-b5ee-4361-9017-959f465f16b1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:18.184391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782a6b-e32b-4f6b-942a-5c3a8753cfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:18.185548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782a6b-e32b-4f6b-942a-5c3a8753cfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.185814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782a6b-e32b-4f6b-942a-5c3a8753cfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.186072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782a6b-e32b-4f6b-942a-5c3a8753cfd7 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.186790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782a6b-e32b-4f6b-942a-5c3a8753cfd7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:18.187080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782a6b-e32b-4f6b-942a-5c3a8753cfd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:18.188498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782a6b-e32b-4f6b-942a-5c3a8753cfd7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:18.215349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677633ba-be9c-4b02-92b6-3e35e5012687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:18.216365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677633ba-be9c-4b02-92b6-3e35e5012687 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.217352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677633ba-be9c-4b02-92b6-3e35e5012687 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.217819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677633ba-be9c-4b02-92b6-3e35e5012687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.227091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 311/619] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 17:51:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:18.230014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677633ba-be9c-4b02-92b6-3e35e5012687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:18.231705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677633ba-be9c-4b02-92b6-3e35e5012687 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:18.233161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782a6b-e32b-4f6b-942a-5c3a8753cfd7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-677633ba-be9c-4b02-92b6-3e35e5012687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:18.260584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:18.261630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:18.262538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:18.262538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:18.262538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:18.265393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 309/620] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:18.273612 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677633ba-be9c-4b02-92b6-3e35e5012687 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:18.284872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:18.368890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:18.447780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782a6b-e32b-4f6b-942a-5c3a8753cfd7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:18.481873 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677633ba-be9c-4b02-92b6-3e35e5012687 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:18.504071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c2f53-74a6-43c3-bac0-675e8ddbedec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:18.505002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c2f53-74a6-43c3-bac0-675e8ddbedec None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.505280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c2f53-74a6-43c3-bac0-675e8ddbedec None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.505720 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c2f53-74a6-43c3-bac0-675e8ddbedec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.517452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a377f-843e-415b-94db-ebaaaed50146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:18.518041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a377f-843e-415b-94db-ebaaaed50146 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.518161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c2f53-74a6-43c3-bac0-675e8ddbedec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:18.518321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a377f-843e-415b-94db-ebaaaed50146 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.518575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a377f-843e-415b-94db-ebaaaed50146 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.519208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a377f-843e-415b-94db-ebaaaed50146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:18.519500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a377f-843e-415b-94db-ebaaaed50146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:18.519600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c2f53-74a6-43c3-bac0-675e8ddbedec None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:18.520855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a377f-843e-415b-94db-ebaaaed50146 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:18.533325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 310/621] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:18.534570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18b43-c605-45ff-a1b1-e84afd78ed39 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.534903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18b43-c605-45ff-a1b1-e84afd78ed39 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.535167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18b43-c605-45ff-a1b1-e84afd78ed39 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.536363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 311/622] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-0c4c2f53-74a6-43c3-bac0-675e8ddbedec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:18.540985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:18.541924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:18.542829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:18.542829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:18.542829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:18.542999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d6cab-b4ae-442b-baa6-f096565b2361 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.542999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d6cab-b4ae-442b-baa6-f096565b2361 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.542999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d6cab-b4ae-442b-baa6-f096565b2361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.546665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 312/623] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:18.547465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c2f53-74a6-43c3-bac0-675e8ddbedec None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:18.561731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6c8d6cab-b4ae-442b-baa6-f096565b2361 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:18.563010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6c8d6cab-b4ae-442b-baa6-f096565b2361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:18.633666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:18.644592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 312/624] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:51:18.645682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d6cab-b4ae-442b-baa6-f096565b2361 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:18.667266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c2f53-74a6-43c3-bac0-675e8ddbedec None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:18.682342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:18.711661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52092131-7e6e-478d-b455-f9230c568ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:18.712208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52092131-7e6e-478d-b455-f9230c568ccc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.712484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52092131-7e6e-478d-b455-f9230c568ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.712797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52092131-7e6e-478d-b455-f9230c568ccc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.722708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52092131-7e6e-478d-b455-f9230c568ccc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:18.725246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52092131-7e6e-478d-b455-f9230c568ccc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-52092131-7e6e-478d-b455-f9230c568ccc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:18.744184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:18.745238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:18.746207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 313/625] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 17:51:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:18.746803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52092131-7e6e-478d-b455-f9230c568ccc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:18.782595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d6cab-b4ae-442b-baa6-f096565b2361 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:18.824182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392751-30a4-4476-bc67-a81f8e7fae76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:18.825239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392751-30a4-4476-bc67-a81f8e7fae76 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.825769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392751-30a4-4476-bc67-a81f8e7fae76 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.826335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392751-30a4-4476-bc67-a81f8e7fae76 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.827179 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27392751-30a4-4476-bc67-a81f8e7fae76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:18.827679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27392751-30a4-4476-bc67-a81f8e7fae76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:18.829869 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27392751-30a4-4476-bc67-a81f8e7fae76 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:18.847739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 313/626] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:18.847739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392751-30a4-4476-bc67-a81f8e7fae76 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:18.874732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52092131-7e6e-478d-b455-f9230c568ccc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:18.906617 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:18.911672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ad64d-457c-4900-b2b1-33c7baeb44ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:18.912410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ad64d-457c-4900-b2b1-33c7baeb44ae None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:18.912824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ad64d-457c-4900-b2b1-33c7baeb44ae None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:18.913187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ad64d-457c-4900-b2b1-33c7baeb44ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:18.924605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ad64d-457c-4900-b2b1-33c7baeb44ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f45ad64d-457c-4900-b2b1-33c7baeb44ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:18.930189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:18.931394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:18.932592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 314/627] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:18.933332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ad64d-457c-4900-b2b1-33c7baeb44ae None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:18.945421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:18.968590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392751-30a4-4476-bc67-a81f8e7fae76 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.007031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17081-ee9d-47d0-9458-374b8c724293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:19.007922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17081-ee9d-47d0-9458-374b8c724293 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.008400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17081-ee9d-47d0-9458-374b8c724293 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.008876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17081-ee9d-47d0-9458-374b8c724293 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.009811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17081-ee9d-47d0-9458-374b8c724293 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:19.010400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17081-ee9d-47d0-9458-374b8c724293 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:19.012216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17081-ee9d-47d0-9458-374b8c724293 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:19.036428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 314/628] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 17:51:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:19.037046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ad64d-457c-4900-b2b1-33c7baeb44ae None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.037833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17081-ee9d-47d0-9458-374b8c724293 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:19.072213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c521519-2104-4add-a849-2afa2af5badf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:19.073014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c521519-2104-4add-a849-2afa2af5badf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.073385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c521519-2104-4add-a849-2afa2af5badf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.073750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c521519-2104-4add-a849-2afa2af5badf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.080857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c521519-2104-4add-a849-2afa2af5badf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:19.082453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c521519-2104-4add-a849-2afa2af5badf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-0c521519-2104-4add-a849-2afa2af5badf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:19.094256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:19.095225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:19.096479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:19.096479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:19.096479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:19.096479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:19.096479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:19.096479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:19.096479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:19.096479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:19.096479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 315/629] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:19.101279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c521519-2104-4add-a849-2afa2af5badf None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:19.151679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:19.155463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17081-ee9d-47d0-9458-374b8c724293 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.192043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585ab51-8472-4421-9cf5-c164c5b21d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:19.193004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585ab51-8472-4421-9cf5-c164c5b21d6d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.193502 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585ab51-8472-4421-9cf5-c164c5b21d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.193953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585ab51-8472-4421-9cf5-c164c5b21d6d None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.194840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585ab51-8472-4421-9cf5-c164c5b21d6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:19.195396 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585ab51-8472-4421-9cf5-c164c5b21d6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:19.197934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585ab51-8472-4421-9cf5-c164c5b21d6d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:19.203416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c521519-2104-4add-a849-2afa2af5badf None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.204493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:19.208089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 315/630] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:19.212984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585ab51-8472-4421-9cf5-c164c5b21d6d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:19.241271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf613a-f4ad-45eb-a085-540588e9ff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:19.241801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf613a-f4ad-45eb-a085-540588e9ff0c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.242070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf613a-f4ad-45eb-a085-540588e9ff0c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.242328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf613a-f4ad-45eb-a085-540588e9ff0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.309912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf613a-f4ad-45eb-a085-540588e9ff0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:19.311348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf613a-f4ad-45eb-a085-540588e9ff0c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:19.318023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585ab51-8472-4421-9cf5-c164c5b21d6d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.353686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249c6de-b628-474a-a5e2-682ce353b53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:19.354538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249c6de-b628-474a-a5e2-682ce353b53e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.354985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249c6de-b628-474a-a5e2-682ce353b53e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.355510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249c6de-b628-474a-a5e2-682ce353b53e None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.356197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249c6de-b628-474a-a5e2-682ce353b53e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:19.356665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249c6de-b628-474a-a5e2-682ce353b53e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:19.358091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249c6de-b628-474a-a5e2-682ce353b53e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:19.359799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:19.372649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 316/631] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:19.374226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367f521-4022-4b24-8b06-e9123fc8d876 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.374514 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367f521-4022-4b24-8b06-e9123fc8d876 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.374784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367f521-4022-4b24-8b06-e9123fc8d876 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.376167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 317/632] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:19.376870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367f521-4022-4b24-8b06-e9123fc8d876 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:19.421526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 316/633] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:19.422415 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf613a-f4ad-45eb-a085-540588e9ff0c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:19.440744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:19.485264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367f521-4022-4b24-8b06-e9123fc8d876 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.521812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6809061-13c3-4d62-8c71-32e6e8249a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:19.522349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6809061-13c3-4d62-8c71-32e6e8249a55 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.522620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6809061-13c3-4d62-8c71-32e6e8249a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.522884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6809061-13c3-4d62-8c71-32e6e8249a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.526300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf613a-f4ad-45eb-a085-540588e9ff0c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.533668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6809061-13c3-4d62-8c71-32e6e8249a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:19.535089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6809061-13c3-4d62-8c71-32e6e8249a55 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b6809061-13c3-4d62-8c71-32e6e8249a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:19.549215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:19.550178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:19.551060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:19.551060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:19.551060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:19.554822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 318/634] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:19.556151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4f2ea-cfb1-4e1b-ba5d-1dcb5afc9fc9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.556395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4f2ea-cfb1-4e1b-ba5d-1dcb5afc9fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.556611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4f2ea-cfb1-4e1b-ba5d-1dcb5afc9fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.563112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ad8a-fc8a-4ef6-9366-7a4ef0ade9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:19.564605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ad8a-fc8a-4ef6-9366-7a4ef0ade9a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.564883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ad8a-fc8a-4ef6-9366-7a4ef0ade9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.565151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ad8a-fc8a-4ef6-9366-7a4ef0ade9a9 None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/3375c3134ebe473cb5da29f083321cdb/roles/2978a8a56cd84b4b852dc6738c5c54c3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.574560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-38d4f2ea-cfb1-4e1b-ba5d-1dcb5afc9fc9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:19.576324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-38d4f2ea-cfb1-4e1b-ba5d-1dcb5afc9fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:19.580992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7ad8a-fc8a-4ef6-9366-7a4ef0ade9a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=3375c3134ebe473cb5da29f083321cdb, role_id=2978a8a56cd84b4b852dc6738c5c54c3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:19.582402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7ad8a-fc8a-4ef6-9366-7a4ef0ade9a9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:19.643528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 319/635] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:19.644348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4f2ea-cfb1-4e1b-ba5d-1dcb5afc9fc9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:19.660327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:19.666891 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 317/636] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:51:19 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/3375c3134ebe473cb5da29f083321cdb/roles/2978a8a56cd84b4b852dc6738c5c54c3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:51:19.667687 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ad8a-fc8a-4ef6-9366-7a4ef0ade9a9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:19.705366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:19.782332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4f2ea-cfb1-4e1b-ba5d-1dcb5afc9fc9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.828728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ad8a-fc8a-4ef6-9366-7a4ef0ade9a9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.829225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcab60-207c-4cae-978b-5e77f1cb709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:19.829874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcab60-207c-4cae-978b-5e77f1cb709d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.830149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcab60-207c-4cae-978b-5e77f1cb709d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.830414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcab60-207c-4cae-978b-5e77f1cb709d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.841797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dcab60-207c-4cae-978b-5e77f1cb709d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:19.843259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dcab60-207c-4cae-978b-5e77f1cb709d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b1dcab60-207c-4cae-978b-5e77f1cb709d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:19.858138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:19.859144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:19.860781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:19.860781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:19.860781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:19.860781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 320/637] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:19.860781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcab60-207c-4cae-978b-5e77f1cb709d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:19.867517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f5573-9dad-48d0-8f62-dc8291d5d243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:19.868123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f5573-9dad-48d0-8f62-dc8291d5d243 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.868399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f5573-9dad-48d0-8f62-dc8291d5d243 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.868697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f5573-9dad-48d0-8f62-dc8291d5d243 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.869336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f5573-9dad-48d0-8f62-dc8291d5d243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:19.869633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f5573-9dad-48d0-8f62-dc8291d5d243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:19.871002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f5573-9dad-48d0-8f62-dc8291d5d243 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:19.883404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 318/638] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:19.884648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e82f12-a806-4130-b767-2261db275cbe None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:19.884933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e82f12-a806-4130-b767-2261db275cbe None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:19.885201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e82f12-a806-4130-b767-2261db275cbe None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:19.886524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 319/639] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:19.887068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e82f12-a806-4130-b767-2261db275cbe None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:19.953827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:19.965599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:19.972440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcab60-207c-4cae-978b-5e77f1cb709d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:19.997710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e82f12-a806-4130-b767-2261db275cbe None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.009329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237c44a-791a-4942-8d79-d61d137acdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.009957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237c44a-791a-4942-8d79-d61d137acdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.010227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237c44a-791a-4942-8d79-d61d137acdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.010491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237c44a-791a-4942-8d79-d61d137acdb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.021476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237c44a-791a-4942-8d79-d61d137acdb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f237c44a-791a-4942-8d79-d61d137acdb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:20.026526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:20.032415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 321/640] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:20.033546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237c44a-791a-4942-8d79-d61d137acdb3 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.040908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a39c05-79de-4ad8-b9e3-16d9bef840ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.041678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a39c05-79de-4ad8-b9e3-16d9bef840ab None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.042131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a39c05-79de-4ad8-b9e3-16d9bef840ab None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.042395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a39c05-79de-4ad8-b9e3-16d9bef840ab None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.043035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a39c05-79de-4ad8-b9e3-16d9bef840ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:20.043331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a39c05-79de-4ad8-b9e3-16d9bef840ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.045253 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a39c05-79de-4ad8-b9e3-16d9bef840ab None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.057453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 320/641] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:20.058696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d77f-a9dc-487a-ac00-2cf154c2d1f1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.058982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d77f-a9dc-487a-ac00-2cf154c2d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.059252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d77f-a9dc-487a-ac00-2cf154c2d1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.077313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2d95d77f-a9dc-487a-ac00-2cf154c2d1f1 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:20.078501 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2d95d77f-a9dc-487a-ac00-2cf154c2d1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:20.142091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237c44a-791a-4942-8d79-d61d137acdb3 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.152544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 321/642] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:20.152544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95d77f-a9dc-487a-ac00-2cf154c2d1f1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.183306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e20332-5643-4228-a5b3-b27565172a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.183926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e20332-5643-4228-a5b3-b27565172a71 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.184215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e20332-5643-4228-a5b3-b27565172a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.185056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e20332-5643-4228-a5b3-b27565172a71 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.192658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e20332-5643-4228-a5b3-b27565172a71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.194087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e20332-5643-4228-a5b3-b27565172a71 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.205287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-e5e20332-5643-4228-a5b3-b27565172a71 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:20.205421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:20.206379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 322/643] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:20.207393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e20332-5643-4228-a5b3-b27565172a71 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.224724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:20.264905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95d77f-a9dc-487a-ac00-2cf154c2d1f1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.299537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07673a-c1a6-4781-9634-48fc1536e281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.300169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07673a-c1a6-4781-9634-48fc1536e281 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.300441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07673a-c1a6-4781-9634-48fc1536e281 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.300734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07673a-c1a6-4781-9634-48fc1536e281 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.301393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07673a-c1a6-4781-9634-48fc1536e281 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:20.301684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07673a-c1a6-4781-9634-48fc1536e281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.303027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07673a-c1a6-4781-9634-48fc1536e281 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.315757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 322/644] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:20.315757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07673a-c1a6-4781-9634-48fc1536e281 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.321793 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e20332-5643-4228-a5b3-b27565172a71 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.358827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9176f58-5753-430a-974c-568b950a5fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.359457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9176f58-5753-430a-974c-568b950a5fff None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.359737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9176f58-5753-430a-974c-568b950a5fff None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.360006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9176f58-5753-430a-974c-568b950a5fff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.367444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9176f58-5753-430a-974c-568b950a5fff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.369139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9176f58-5753-430a-974c-568b950a5fff None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-a9176f58-5753-430a-974c-568b950a5fff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:20.382544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:20.383461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 323/645] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:20.384579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9176f58-5753-430a-974c-568b950a5fff None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.421453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07673a-c1a6-4781-9634-48fc1536e281 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.433831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:20.454491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbf265-01cb-4bff-8851-0e34c998730b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.455103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbf265-01cb-4bff-8851-0e34c998730b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.455375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbf265-01cb-4bff-8851-0e34c998730b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.455637 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbf265-01cb-4bff-8851-0e34c998730b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.466510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:20.466637 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbf265-01cb-4bff-8851-0e34c998730b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-dacbf265-01cb-4bff-8851-0e34c998730b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:20.471538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:20.472702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:20.473785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:20.474639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 323/646] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:20.475927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbf265-01cb-4bff-8851-0e34c998730b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.496111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9176f58-5753-430a-974c-568b950a5fff None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.531695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d899a93-9601-4153-a9af-397b26752b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.532599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d899a93-9601-4153-a9af-397b26752b87 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.533151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d899a93-9601-4153-a9af-397b26752b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.533673 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d899a93-9601-4153-a9af-397b26752b87 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.534670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d899a93-9601-4153-a9af-397b26752b87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:20.535145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d899a93-9601-4153-a9af-397b26752b87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.537029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d899a93-9601-4153-a9af-397b26752b87 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.555360 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 324/647] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:20.556811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d899a93-9601-4153-a9af-397b26752b87 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.580737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbf265-01cb-4bff-8851-0e34c998730b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.620216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f834f2f-54dd-4143-832b-c35bf7ca1691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.621208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f834f2f-54dd-4143-832b-c35bf7ca1691 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.621691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f834f2f-54dd-4143-832b-c35bf7ca1691 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.622198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f834f2f-54dd-4143-832b-c35bf7ca1691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.634206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f834f2f-54dd-4143-832b-c35bf7ca1691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.635431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f834f2f-54dd-4143-832b-c35bf7ca1691 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4f834f2f-54dd-4143-832b-c35bf7ca1691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:20.649526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:20.650473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:20.651374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:20.651374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:20.651374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:20.651374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 324/648] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:20.651374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f834f2f-54dd-4143-832b-c35bf7ca1691 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d899a93-9601-4153-a9af-397b26752b87 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.678322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:20.698464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:20.704904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfc4b0-a5d8-4098-92cc-de980c3e8bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.705792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfc4b0-a5d8-4098-92cc-de980c3e8bca None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.706274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfc4b0-a5d8-4098-92cc-de980c3e8bca None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.706781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfc4b0-a5d8-4098-92cc-de980c3e8bca None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.707576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfc4b0-a5d8-4098-92cc-de980c3e8bca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:20.707878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfc4b0-a5d8-4098-92cc-de980c3e8bca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.709337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfc4b0-a5d8-4098-92cc-de980c3e8bca None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.721607 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 325/649] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:20.722391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfc4b0-a5d8-4098-92cc-de980c3e8bca None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.757126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f834f2f-54dd-4143-832b-c35bf7ca1691 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.804598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8b2d-8228-4dfa-8729-3ceb3dab6a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.805234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8b2d-8228-4dfa-8729-3ceb3dab6a9d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.805560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8b2d-8228-4dfa-8729-3ceb3dab6a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.805771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8b2d-8228-4dfa-8729-3ceb3dab6a9d None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.806430 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8b2d-8228-4dfa-8729-3ceb3dab6a9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:20.806722 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8b2d-8228-4dfa-8729-3ceb3dab6a9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.808185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8b2d-8228-4dfa-8729-3ceb3dab6a9d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.818030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 325/650] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:20.819505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8b2d-8228-4dfa-8729-3ceb3dab6a9d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:20.843170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfc4b0-a5d8-4098-92cc-de980c3e8bca None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.879661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e9f1-d22c-4f3d-8069-b5ee27c522ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.880257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e9f1-d22c-4f3d-8069-b5ee27c522ad None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.880556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e9f1-d22c-4f3d-8069-b5ee27c522ad None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.880824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e9f1-d22c-4f3d-8069-b5ee27c522ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.918020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:20.933024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8b2d-8228-4dfa-8729-3ceb3dab6a9d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:20.938711 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:20.948062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030e9f1-d22c-4f3d-8069-b5ee27c522ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.949621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030e9f1-d22c-4f3d-8069-b5ee27c522ad None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.966510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28ca22-a24a-4090-ad22-e12d4082931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:20.967099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28ca22-a24a-4090-ad22-e12d4082931a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:20.967369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28ca22-a24a-4090-ad22-e12d4082931a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:20.967656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28ca22-a24a-4090-ad22-e12d4082931a None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:20.968677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28ca22-a24a-4090-ad22-e12d4082931a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:20.969243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28ca22-a24a-4090-ad22-e12d4082931a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:20.970960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28ca22-a24a-4090-ad22-e12d4082931a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:20.982962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 326/651] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:20.983784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28ca22-a24a-4090-ad22-e12d4082931a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.058620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 326/652] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:21.059683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e9f1-d22c-4f3d-8069-b5ee27c522ad None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.086823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28ca22-a24a-4090-ad22-e12d4082931a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:21.127017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcd6f8-a65c-4a38-8f45-0499cb63632d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:21.127775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcd6f8-a65c-4a38-8f45-0499cb63632d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.128044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcd6f8-a65c-4a38-8f45-0499cb63632d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.128380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcd6f8-a65c-4a38-8f45-0499cb63632d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.129104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcd6f8-a65c-4a38-8f45-0499cb63632d None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=c496da229db34f90a4b3fdcfa582dca8) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:21.129401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcd6f8-a65c-4a38-8f45-0499cb63632d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:21.130807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcd6f8-a65c-4a38-8f45-0499cb63632d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:21.139192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 327/653] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:20 2022] GET /identity/v3/role_assignments?user.id=c496da229db34f90a4b3fdcfa582dca8&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:21.140400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcd6f8-a65c-4a38-8f45-0499cb63632d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.155708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:21.168904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:21.179470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e9f1-d22c-4f3d-8069-b5ee27c522ad None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:21.227560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7ede6-e9ba-4048-902d-160d6ae57f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:21.228331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7ede6-e9ba-4048-902d-160d6ae57f21 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.228743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7ede6-e9ba-4048-902d-160d6ae57f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.229103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7ede6-e9ba-4048-902d-160d6ae57f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.241561 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7ede6-e9ba-4048-902d-160d6ae57f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:21.243839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7ede6-e9ba-4048-902d-160d6ae57f21 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:21.260420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcd6f8-a65c-4a38-8f45-0499cb63632d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-53f7ede6-e9ba-4048-902d-160d6ae57f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:21.265988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:21.267036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:21.268853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:21.268853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:21.268853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:21.268853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 327/654] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:21.268853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7ede6-e9ba-4048-902d-160d6ae57f21 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.302521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7a27d-a801-420c-9f43-72a930263fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:21.303226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7a27d-a801-420c-9f43-72a930263fce None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.303498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7a27d-a801-420c-9f43-72a930263fce None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.303760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7a27d-a801-420c-9f43-72a930263fce None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.312864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7a27d-a801-420c-9f43-72a930263fce None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:21.314235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7a27d-a801-420c-9f43-72a930263fce None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-aba7a27d-a801-420c-9f43-72a930263fce None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:21.328525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:21.329793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 328/655] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:21.332134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7a27d-a801-420c-9f43-72a930263fce None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.426695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:21.429734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7ede6-e9ba-4048-902d-160d6ae57f21 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:21.465311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7a27d-a801-420c-9f43-72a930263fce None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:21.472849 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08d3ac-4a3f-4012-9200-129a0ec84396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:21.473847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08d3ac-4a3f-4012-9200-129a0ec84396 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.474319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08d3ac-4a3f-4012-9200-129a0ec84396 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.474785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08d3ac-4a3f-4012-9200-129a0ec84396 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.487498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08d3ac-4a3f-4012-9200-129a0ec84396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:21.489371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08d3ac-4a3f-4012-9200-129a0ec84396 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:21.492788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:21.505359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133ddf5-f6e4-4474-9cb5-be8beb84f3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:21.508923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133ddf5-f6e4-4474-9cb5-be8beb84f3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.508923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133ddf5-f6e4-4474-9cb5-be8beb84f3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.508923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133ddf5-f6e4-4474-9cb5-be8beb84f3b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.508923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133ddf5-f6e4-4474-9cb5-be8beb84f3b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:21.508923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133ddf5-f6e4-4474-9cb5-be8beb84f3b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5b08d3ac-4a3f-4012-9200-129a0ec84396 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:21.509307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:21.510178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:21.511077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:21.511077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:21.511077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:21.511255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133ddf5-f6e4-4474-9cb5-be8beb84f3b8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:21.511334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 328/656] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:21.511334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08d3ac-4a3f-4012-9200-129a0ec84396 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.527930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 329/657] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:21.529248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133ddf5-f6e4-4474-9cb5-be8beb84f3b8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.643031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08d3ac-4a3f-4012-9200-129a0ec84396 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:21.648867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133ddf5-f6e4-4474-9cb5-be8beb84f3b8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:21.686228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce94278-4f2d-46a2-ad1f-8b680a03df6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:21.686829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce94278-4f2d-46a2-ad1f-8b680a03df6b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.687101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce94278-4f2d-46a2-ad1f-8b680a03df6b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.687363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce94278-4f2d-46a2-ad1f-8b680a03df6b None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.688171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:21.688660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce94278-4f2d-46a2-ad1f-8b680a03df6b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:21.689604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce94278-4f2d-46a2-ad1f-8b680a03df6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:21.691832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce94278-4f2d-46a2-ad1f-8b680a03df6b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:21.696559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30108f-bbf2-4971-a582-0cc59fd2a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:21.703618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30108f-bbf2-4971-a582-0cc59fd2a14f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.703618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30108f-bbf2-4971-a582-0cc59fd2a14f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.703618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30108f-bbf2-4971-a582-0cc59fd2a14f None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.703618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30108f-bbf2-4971-a582-0cc59fd2a14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:21.703618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30108f-bbf2-4971-a582-0cc59fd2a14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:21.703618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30108f-bbf2-4971-a582-0cc59fd2a14f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:21.712885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 330/658] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:21.721059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 329/659] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:21.721225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30108f-bbf2-4971-a582-0cc59fd2a14f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.721319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d94ae-a645-4fed-8983-d84a0649e75d None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.721319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d94ae-a645-4fed-8983-d84a0649e75d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.721319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d94ae-a645-4fed-8983-d84a0649e75d None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.721319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 330/660] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:21.721319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d94ae-a645-4fed-8983-d84a0649e75d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.779844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:21.833419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d94ae-a645-4fed-8983-d84a0649e75d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:21.835103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30108f-bbf2-4971-a582-0cc59fd2a14f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:21.867989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238b856-1558-46d2-bbd8-085a59431d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:21.868633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238b856-1558-46d2-bbd8-085a59431d44 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.868908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238b856-1558-46d2-bbd8-085a59431d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.869174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238b856-1558-46d2-bbd8-085a59431d44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.875904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238b856-1558-46d2-bbd8-085a59431d44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:21.877370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238b856-1558-46d2-bbd8-085a59431d44 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:21.878150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69634f0a-a3ef-415d-90ce-e8aba9540819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:21.879131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69634f0a-a3ef-415d-90ce-e8aba9540819 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.879579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69634f0a-a3ef-415d-90ce-e8aba9540819 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.880262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69634f0a-a3ef-415d-90ce-e8aba9540819 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.888038 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69634f0a-a3ef-415d-90ce-e8aba9540819 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f238b856-1558-46d2-bbd8-085a59431d44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:21.888369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:21.889263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:21.890182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:21.890182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:21.890182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:21.890182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:21.890182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:21.890182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:21.890182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:21.890182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:21.891092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69634f0a-a3ef-415d-90ce-e8aba9540819 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:21.894334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 331/661] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:21.895317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039445ba-dac9-4b74-832b-62a46b0f24fd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:21.895600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039445ba-dac9-4b74-832b-62a46b0f24fd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:21.895871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039445ba-dac9-4b74-832b-62a46b0f24fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-69634f0a-a3ef-415d-90ce-e8aba9540819 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:21.905153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:21.906132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 17:51:21.907092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:21.913389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 331/662] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:21.914321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69634f0a-a3ef-415d-90ce-e8aba9540819 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:21.915235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-039445ba-dac9-4b74-832b-62a46b0f24fd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:21.916430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-039445ba-dac9-4b74-832b-62a46b0f24fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:21.966954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:21.986531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 332/663] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:21.987675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039445ba-dac9-4b74-832b-62a46b0f24fd None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:22.020975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69634f0a-a3ef-415d-90ce-e8aba9540819 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:22.042839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:22.056543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5ad76-3ce4-4868-91fe-13c03b4ac3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:22.057234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5ad76-3ce4-4868-91fe-13c03b4ac3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:22.057517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5ad76-3ce4-4868-91fe-13c03b4ac3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:22.057780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5ad76-3ce4-4868-91fe-13c03b4ac3b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:22.100750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039445ba-dac9-4b74-832b-62a46b0f24fd None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:22.127046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5ad76-3ce4-4868-91fe-13c03b4ac3b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:22.128520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5ad76-3ce4-4868-91fe-13c03b4ac3b0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:22.136237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40b859-3869-4ab6-ac91-1ea28292d15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:22.136848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40b859-3869-4ab6-ac91-1ea28292d15c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:22.137117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40b859-3869-4ab6-ac91-1ea28292d15c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:22.139198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40b859-3869-4ab6-ac91-1ea28292d15c None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:22.140004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40b859-3869-4ab6-ac91-1ea28292d15c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:22.140309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40b859-3869-4ab6-ac91-1ea28292d15c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:22.141766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40b859-3869-4ab6-ac91-1ea28292d15c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:22.154870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 333/664] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:22.155593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40b859-3869-4ab6-ac91-1ea28292d15c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:22.182197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:22.237304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 332/665] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:22.238334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5ad76-3ce4-4868-91fe-13c03b4ac3b0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:22.267435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40b859-3869-4ab6-ac91-1ea28292d15c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:22.269877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:22.302598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee214aee-d7f0-43db-ab99-0a450f562250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:22.303462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee214aee-d7f0-43db-ab99-0a450f562250 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:22.303846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee214aee-d7f0-43db-ab99-0a450f562250 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:22.304121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee214aee-d7f0-43db-ab99-0a450f562250 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:22.304929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee214aee-d7f0-43db-ab99-0a450f562250 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:22.305257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee214aee-d7f0-43db-ab99-0a450f562250 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:22.306646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee214aee-d7f0-43db-ab99-0a450f562250 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:22.329419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 334/666] 10.222.0.50 () {56 vars in 1139 bytes} [Mon Aug 8 17:51:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:22.329419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee214aee-d7f0-43db-ab99-0a450f562250 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:22.346345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5ad76-3ce4-4868-91fe-13c03b4ac3b0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:22.389882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cd012-9d84-44ed-8d2e-7ac7004d9571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:22.391049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cd012-9d84-44ed-8d2e-7ac7004d9571 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:22.391745 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cd012-9d84-44ed-8d2e-7ac7004d9571 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:22.392463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cd012-9d84-44ed-8d2e-7ac7004d9571 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:22.405642 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cd012-9d84-44ed-8d2e-7ac7004d9571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f16cd012-9d84-44ed-8d2e-7ac7004d9571 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:22.412525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:22.413670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:22.414552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 333/667] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:22.415400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cd012-9d84-44ed-8d2e-7ac7004d9571 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:22.430487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:22.444619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee214aee-d7f0-43db-ab99-0a450f562250 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:22.479660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6be6b-31ab-4fbe-8bcb-153943622c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:22.480258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6be6b-31ab-4fbe-8bcb-153943622c1b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:22.480555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6be6b-31ab-4fbe-8bcb-153943622c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:22.480826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6be6b-31ab-4fbe-8bcb-153943622c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:22.481567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6be6b-31ab-4fbe-8bcb-153943622c1b None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=0e95d295337b4472be946d252ac3bd7e) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:22.481868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6be6b-31ab-4fbe-8bcb-153943622c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:22.483285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6be6b-31ab-4fbe-8bcb-153943622c1b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:22.492568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 335/668] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:22 2022] GET /identity/v3/role_assignments?user.id=0e95d295337b4472be946d252ac3bd7e&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:22.492568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6be6b-31ab-4fbe-8bcb-153943622c1b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:22.525629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:22.550687 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cd012-9d84-44ed-8d2e-7ac7004d9571 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:22.601681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2dd27-98dd-48ad-8bfa-d5532664cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:22.602998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2dd27-98dd-48ad-8bfa-d5532664cc5f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:22.603866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2dd27-98dd-48ad-8bfa-d5532664cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:22.604334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2dd27-98dd-48ad-8bfa-d5532664cc5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:22.619935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2dd27-98dd-48ad-8bfa-d5532664cc5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:22.621375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6be6b-31ab-4fbe-8bcb-153943622c1b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:22.622603 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2dd27-98dd-48ad-8bfa-d5532664cc5f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6ed2dd27-98dd-48ad-8bfa-d5532664cc5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:22.649220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:22.650223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:22.651113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:22.651113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:22.651113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:22.656655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 334/669] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:22.657639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2dd27-98dd-48ad-8bfa-d5532664cc5f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:22.665925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf3822-6468-4eed-9231-a929f1cc21b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:22.666528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf3822-6468-4eed-9231-a929f1cc21b5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:22.666800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf3822-6468-4eed-9231-a929f1cc21b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:22.667069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf3822-6468-4eed-9231-a929f1cc21b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:22.731916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:22.743868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf3822-6468-4eed-9231-a929f1cc21b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:22.746247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf3822-6468-4eed-9231-a929f1cc21b5 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:22.779557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:22.789292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2dd27-98dd-48ad-8bfa-d5532664cc5f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:22.838077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d432aab-5061-458a-b1bf-326ccc497983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:22.839333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d432aab-5061-458a-b1bf-326ccc497983 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:22.839616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d432aab-5061-458a-b1bf-326ccc497983 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:22.839885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d432aab-5061-458a-b1bf-326ccc497983 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:22.840942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d432aab-5061-458a-b1bf-326ccc497983 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:22.841241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d432aab-5061-458a-b1bf-326ccc497983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:22.842608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d432aab-5061-458a-b1bf-326ccc497983 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:22.856452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 335/670] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:22.857829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d432aab-5061-458a-b1bf-326ccc497983 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:22.884062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 336/671] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:22.885091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf3822-6468-4eed-9231-a929f1cc21b5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:22.979779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d432aab-5061-458a-b1bf-326ccc497983 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.004772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:23.007121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf3822-6468-4eed-9231-a929f1cc21b5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.016568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643324e-93e8-4503-8c37-bc01480add86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.017381 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643324e-93e8-4503-8c37-bc01480add86 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.024571 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643324e-93e8-4503-8c37-bc01480add86 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.024571 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643324e-93e8-4503-8c37-bc01480add86 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.024571 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643324e-93e8-4503-8c37-bc01480add86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:23.024571 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643324e-93e8-4503-8c37-bc01480add86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.024571 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643324e-93e8-4503-8c37-bc01480add86 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.035430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:23.037073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 336/672] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:23.037985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643324e-93e8-4503-8c37-bc01480add86 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.045982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6585d-c703-492c-8b46-308991df2a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.046917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6585d-c703-492c-8b46-308991df2a4f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.047428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6585d-c703-492c-8b46-308991df2a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.047908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6585d-c703-492c-8b46-308991df2a4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.058768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6585d-c703-492c-8b46-308991df2a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.061074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6585d-c703-492c-8b46-308991df2a4f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-0dc6585d-c703-492c-8b46-308991df2a4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:23.076601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:23.077596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:23.078605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:23.078605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:23.078605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:23.080954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 337/673] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:23.082061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6585d-c703-492c-8b46-308991df2a4f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.159612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643324e-93e8-4503-8c37-bc01480add86 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.199358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd4a77-9907-4927-a299-be8f489fe018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.200411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd4a77-9907-4927-a299-be8f489fe018 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.200958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd4a77-9907-4927-a299-be8f489fe018 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.201439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd4a77-9907-4927-a299-be8f489fe018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.210264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6585d-c703-492c-8b46-308991df2a4f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.213288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd4a77-9907-4927-a299-be8f489fe018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.215002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd4a77-9907-4927-a299-be8f489fe018 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-36bd4a77-9907-4927-a299-be8f489fe018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:23.230201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:23.231759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:23.232683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:23.232683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:23.232683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:23.239098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 337/674] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:23.240194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd4a77-9907-4927-a299-be8f489fe018 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.250905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef349c-ff11-40d2-b735-b47227bfe35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.251801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef349c-ff11-40d2-b735-b47227bfe35b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.252289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef349c-ff11-40d2-b735-b47227bfe35b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.252954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef349c-ff11-40d2-b735-b47227bfe35b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.260589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef349c-ff11-40d2-b735-b47227bfe35b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.262248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef349c-ff11-40d2-b735-b47227bfe35b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-bfef349c-ff11-40d2-b735-b47227bfe35b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:23.274487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:23.275418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:23.276315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:23.276315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:23.276315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:23.276315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:23.276315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:23.276315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:23.276315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:23.276315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:23.278866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 338/675] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:23.279854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef349c-ff11-40d2-b735-b47227bfe35b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.284886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:23.327434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:23.368177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd4a77-9907-4927-a299-be8f489fe018 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.390225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef349c-ff11-40d2-b735-b47227bfe35b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.402109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1d8dc-5cdc-4364-b596-0ecf8ea7bd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.402897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1d8dc-5cdc-4364-b596-0ecf8ea7bd46 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.403817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1d8dc-5cdc-4364-b596-0ecf8ea7bd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.404184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1d8dc-5cdc-4364-b596-0ecf8ea7bd46 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.404963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1d8dc-5cdc-4364-b596-0ecf8ea7bd46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:23.405405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1d8dc-5cdc-4364-b596-0ecf8ea7bd46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.406988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1d8dc-5cdc-4364-b596-0ecf8ea7bd46 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.424586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 338/676] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:23.425483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1d8dc-5cdc-4364-b596-0ecf8ea7bd46 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.425595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b11f85-e0f4-4389-b5d9-a42e7f1c888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.426255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b11f85-e0f4-4389-b5d9-a42e7f1c888e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.426592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b11f85-e0f4-4389-b5d9-a42e7f1c888e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.426954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b11f85-e0f4-4389-b5d9-a42e7f1c888e None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.427633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b11f85-e0f4-4389-b5d9-a42e7f1c888e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:23.428027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b11f85-e0f4-4389-b5d9-a42e7f1c888e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.429452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b11f85-e0f4-4389-b5d9-a42e7f1c888e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.441899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 339/677] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:23.443152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d9116-1911-45c5-883d-163d4b2f484c None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.443496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d9116-1911-45c5-883d-163d4b2f484c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.443826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d9116-1911-45c5-883d-163d4b2f484c None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.445257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 340/678] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:23.449725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d9116-1911-45c5-883d-163d4b2f484c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.532278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:23.538289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1d8dc-5cdc-4364-b596-0ecf8ea7bd46 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.548928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:23.553792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d9116-1911-45c5-883d-163d4b2f484c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.573900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd81599-29ad-45b1-b55d-e1c60d690a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.574495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd81599-29ad-45b1-b55d-e1c60d690a16 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.574772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd81599-29ad-45b1-b55d-e1c60d690a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.575059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd81599-29ad-45b1-b55d-e1c60d690a16 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.575733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd81599-29ad-45b1-b55d-e1c60d690a16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:23.576028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd81599-29ad-45b1-b55d-e1c60d690a16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.577573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd81599-29ad-45b1-b55d-e1c60d690a16 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.587498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e5c00-6f50-49a2-b738-613178ff616b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.588183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e5c00-6f50-49a2-b738-613178ff616b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.588544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e5c00-6f50-49a2-b738-613178ff616b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.588885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e5c00-6f50-49a2-b738-613178ff616b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.595509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 339/679] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:23.595509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd81599-29ad-45b1-b55d-e1c60d690a16 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.601575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e5c00-6f50-49a2-b738-613178ff616b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.603101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e5c00-6f50-49a2-b738-613178ff616b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-cb6e5c00-6f50-49a2-b738-613178ff616b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:23.616856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:23.617979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:23.619140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:23.619684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 341/680] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:23.620777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040de11-065c-4e29-8b1f-01a5ccdcb464 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.621052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040de11-065c-4e29-8b1f-01a5ccdcb464 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.621317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040de11-065c-4e29-8b1f-01a5ccdcb464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.638860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e040de11-065c-4e29-8b1f-01a5ccdcb464 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:23.640026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e040de11-065c-4e29-8b1f-01a5ccdcb464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:23.707375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd81599-29ad-45b1-b55d-e1c60d690a16 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.709837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 342/681] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:23.710673 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040de11-065c-4e29-8b1f-01a5ccdcb464 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.756257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db874987-f5d3-4827-ba81-bd775773e9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.757019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db874987-f5d3-4827-ba81-bd775773e9bc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.757210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db874987-f5d3-4827-ba81-bd775773e9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.757481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db874987-f5d3-4827-ba81-bd775773e9bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.765977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db874987-f5d3-4827-ba81-bd775773e9bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.767448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db874987-f5d3-4827-ba81-bd775773e9bc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-db874987-f5d3-4827-ba81-bd775773e9bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:23.780019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:23.781200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 340/682] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:23.782188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db874987-f5d3-4827-ba81-bd775773e9bc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.810186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:23.821821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:23.834132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040de11-065c-4e29-8b1f-01a5ccdcb464 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.868213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5d984-a9ab-498b-959c-62a6cde4dda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.868905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5d984-a9ab-498b-959c-62a6cde4dda0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.869176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5d984-a9ab-498b-959c-62a6cde4dda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.869438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5d984-a9ab-498b-959c-62a6cde4dda0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.870147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5d984-a9ab-498b-959c-62a6cde4dda0 None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=5b4aef937fb047f6869a156a68e7bf2f) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:23.870602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5d984-a9ab-498b-959c-62a6cde4dda0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.872225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5d984-a9ab-498b-959c-62a6cde4dda0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.888727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 343/683] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/role_assignments?user.id=5b4aef937fb047f6869a156a68e7bf2f&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:23.889796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5d984-a9ab-498b-959c-62a6cde4dda0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:23.901065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db874987-f5d3-4827-ba81-bd775773e9bc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:23.942105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bb341-740f-416f-85a8-6d10e99aec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:23.942819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bb341-740f-416f-85a8-6d10e99aec17 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:23.943219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bb341-740f-416f-85a8-6d10e99aec17 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:23.943560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bb341-740f-416f-85a8-6d10e99aec17 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:23.944489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bb341-740f-416f-85a8-6d10e99aec17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:23.944950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bb341-740f-416f-85a8-6d10e99aec17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:23.946676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bb341-740f-416f-85a8-6d10e99aec17 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:23.975141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 341/684] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:23.976224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bb341-740f-416f-85a8-6d10e99aec17 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:24.005584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5d984-a9ab-498b-959c-62a6cde4dda0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:24.047256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be094d43-ac1d-4efc-8c5f-07d15cc59dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:24.048213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be094d43-ac1d-4efc-8c5f-07d15cc59dcb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.048711 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be094d43-ac1d-4efc-8c5f-07d15cc59dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.049174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be094d43-ac1d-4efc-8c5f-07d15cc59dcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.061601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be094d43-ac1d-4efc-8c5f-07d15cc59dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-be094d43-ac1d-4efc-8c5f-07d15cc59dcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:24.066611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:24.067785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:24.068868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:24.076845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 344/685] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:24.078448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:24.079815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be094d43-ac1d-4efc-8c5f-07d15cc59dcb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:24.097783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:24.112242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bb341-740f-416f-85a8-6d10e99aec17 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:24.154177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5639f5-f775-4260-882d-080dcfc5d39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:24.154791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5639f5-f775-4260-882d-080dcfc5d39a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.155379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5639f5-f775-4260-882d-080dcfc5d39a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.155797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5639f5-f775-4260-882d-080dcfc5d39a None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.156647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5639f5-f775-4260-882d-080dcfc5d39a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:24.157324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5639f5-f775-4260-882d-080dcfc5d39a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:24.159445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5639f5-f775-4260-882d-080dcfc5d39a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:24.175380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 342/686] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:24.176465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5639f5-f775-4260-882d-080dcfc5d39a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:24.197646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be094d43-ac1d-4efc-8c5f-07d15cc59dcb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:24.238929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b65c1b-2979-46e4-926c-72beb5b87beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:24.239697 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b65c1b-2979-46e4-926c-72beb5b87beb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.240072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b65c1b-2979-46e4-926c-72beb5b87beb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.240442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b65c1b-2979-46e4-926c-72beb5b87beb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.294478 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5639f5-f775-4260-882d-080dcfc5d39a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:24.329367 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b65c1b-2979-46e4-926c-72beb5b87beb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:24.331056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b65c1b-2979-46e4-926c-72beb5b87beb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:24.331860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a68ae-a8e3-4d9a-bd86-ae004c05b4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:24.332591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a68ae-a8e3-4d9a-bd86-ae004c05b4de None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.333103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a68ae-a8e3-4d9a-bd86-ae004c05b4de None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.333517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a68ae-a8e3-4d9a-bd86-ae004c05b4de None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.334343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a68ae-a8e3-4d9a-bd86-ae004c05b4de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:24.334741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a68ae-a8e3-4d9a-bd86-ae004c05b4de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:24.336174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a68ae-a8e3-4d9a-bd86-ae004c05b4de None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:24.345486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 343/687] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:24.346352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:24.347364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a68ae-a8e3-4d9a-bd86-ae004c05b4de None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:24.350135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:24.447825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 345/688] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:24.448708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b65c1b-2979-46e4-926c-72beb5b87beb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:24.504380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a68ae-a8e3-4d9a-bd86-ae004c05b4de None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:24.550663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228b4eb-0a2b-475b-bca1-4d93e856dd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:24.551262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228b4eb-0a2b-475b-bca1-4d93e856dd2a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.551531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228b4eb-0a2b-475b-bca1-4d93e856dd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.551793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228b4eb-0a2b-475b-bca1-4d93e856dd2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.552523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228b4eb-0a2b-475b-bca1-4d93e856dd2a None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=95f02466042946ee96e37c5adc27b01d) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:24.552817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228b4eb-0a2b-475b-bca1-4d93e856dd2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:24.554339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228b4eb-0a2b-475b-bca1-4d93e856dd2a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:24.561736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b65c1b-2979-46e4-926c-72beb5b87beb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:24.564793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 344/689] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:24 2022] GET /identity/v3/role_assignments?user.id=95f02466042946ee96e37c5adc27b01d&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:24.565708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228b4eb-0a2b-475b-bca1-4d93e856dd2a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:24.578072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:24.611613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1105206-544a-4f8f-b4fc-25887e65727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:24.612228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1105206-544a-4f8f-b4fc-25887e65727a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.612516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1105206-544a-4f8f-b4fc-25887e65727a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.614589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1105206-544a-4f8f-b4fc-25887e65727a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.634792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1105206-544a-4f8f-b4fc-25887e65727a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:24.637780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1105206-544a-4f8f-b4fc-25887e65727a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b1105206-544a-4f8f-b4fc-25887e65727a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:24.668298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:24.669885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:24.670783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:24.670783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:24.670783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:24.671527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 346/690] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:24.678159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1105206-544a-4f8f-b4fc-25887e65727a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:24.705275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:24.754796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228b4eb-0a2b-475b-bca1-4d93e856dd2a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:24.796113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c8ca0-37a0-4381-9962-819f7d80fc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:24.796784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c8ca0-37a0-4381-9962-819f7d80fc63 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.797028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c8ca0-37a0-4381-9962-819f7d80fc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.797292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c8ca0-37a0-4381-9962-819f7d80fc63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.811260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c8ca0-37a0-4381-9962-819f7d80fc63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:24.812795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c8ca0-37a0-4381-9962-819f7d80fc63 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:24.827217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1105206-544a-4f8f-b4fc-25887e65727a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1f5c8ca0-37a0-4381-9962-819f7d80fc63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:24.830662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:24.831579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:24.832475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:24.832475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:24.832475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:24.832475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 345/691] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:24.834049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a7df-6fc4-4ed0-be68-809d87e8847c None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.834049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a7df-6fc4-4ed0-be68-809d87e8847c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.834049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a7df-6fc4-4ed0-be68-809d87e8847c None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.834676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 346/692] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:24.839866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeb2e0-60d9-4ab1-88cb-fa0429ffcd70 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.840146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeb2e0-60d9-4ab1-88cb-fa0429ffcd70 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.840978 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeb2e0-60d9-4ab1-88cb-fa0429ffcd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.861401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-dfeeb2e0-60d9-4ab1-88cb-fa0429ffcd70 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:24.862661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-dfeeb2e0-60d9-4ab1-88cb-fa0429ffcd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:24.871929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c0629-ff13-4e45-a415-b26c04b24c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:24.872978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c0629-ff13-4e45-a415-b26c04b24c22 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:24.873468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c0629-ff13-4e45-a415-b26c04b24c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:24.874052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c0629-ff13-4e45-a415-b26c04b24c22 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:24.874786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c0629-ff13-4e45-a415-b26c04b24c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:24.875174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c0629-ff13-4e45-a415-b26c04b24c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:24.876657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c0629-ff13-4e45-a415-b26c04b24c22 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:24.889691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 347/693] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:24.890756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c0629-ff13-4e45-a415-b26c04b24c22 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:24.918440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:24.941418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 347/694] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:51:24.948702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeeb2e0-60d9-4ab1-88cb-fa0429ffcd70 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:24.997521 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c0629-ff13-4e45-a415-b26c04b24c22 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:25.012289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:25.058594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341100a-ca7e-466e-85b1-126ab5913dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:25.059243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341100a-ca7e-466e-85b1-126ab5913dac None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:25.059530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341100a-ca7e-466e-85b1-126ab5913dac None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:25.060258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341100a-ca7e-466e-85b1-126ab5913dac None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:25.061146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341100a-ca7e-466e-85b1-126ab5913dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:25.061565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341100a-ca7e-466e-85b1-126ab5913dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:25.063194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341100a-ca7e-466e-85b1-126ab5913dac None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:25.079926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 348/695] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:25.081193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341100a-ca7e-466e-85b1-126ab5913dac None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:25.085780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeeb2e0-60d9-4ab1-88cb-fa0429ffcd70 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:25.134779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a751b7-8284-4392-9e03-0149c778c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:25.135639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a751b7-8284-4392-9e03-0149c778c09d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:25.136056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a751b7-8284-4392-9e03-0149c778c09d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:25.136485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a751b7-8284-4392-9e03-0149c778c09d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:25.146277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a751b7-8284-4392-9e03-0149c778c09d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:25.148647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a751b7-8284-4392-9e03-0149c778c09d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-37a751b7-8284-4392-9e03-0149c778c09d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:25.162027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:25.163217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 348/696] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 17:51:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:25.164278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a751b7-8284-4392-9e03-0149c778c09d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:25.198061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:25.216638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341100a-ca7e-466e-85b1-126ab5913dac None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:25.252449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13308f0c-4e3d-4172-9965-174862132291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:25.254391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13308f0c-4e3d-4172-9965-174862132291 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:25.254851 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13308f0c-4e3d-4172-9965-174862132291 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:25.255268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13308f0c-4e3d-4172-9965-174862132291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:25.267132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13308f0c-4e3d-4172-9965-174862132291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:25.268822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13308f0c-4e3d-4172-9965-174862132291 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:25.270525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a751b7-8284-4392-9e03-0149c778c09d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:25.272012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-13308f0c-4e3d-4172-9965-174862132291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:25.285049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:25.286190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:25.287257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:25.287257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:25.287257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:25.288884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 349/697] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:25.289797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13308f0c-4e3d-4172-9965-174862132291 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:25.305846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4046477-4457-4e03-915f-30aa2483af76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:25.306473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4046477-4457-4e03-915f-30aa2483af76 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:25.306743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4046477-4457-4e03-915f-30aa2483af76 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:25.307105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4046477-4457-4e03-915f-30aa2483af76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:25.315213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4046477-4457-4e03-915f-30aa2483af76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:25.316255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4046477-4457-4e03-915f-30aa2483af76 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c4046477-4457-4e03-915f-30aa2483af76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:25.331776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:25.332899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:25.333853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:25.333853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:25.333853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:25.333853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:25.333853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:25.333853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:25.333853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:25.333853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:25.333853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 349/698] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:25.334339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4046477-4457-4e03-915f-30aa2483af76 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:25.451545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4046477-4457-4e03-915f-30aa2483af76 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:25.494019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262096a-a9c0-4ce8-9aa2-e5c7ec8c0e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:25.494640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262096a-a9c0-4ce8-9aa2-e5c7ec8c0e9f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:25.494916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262096a-a9c0-4ce8-9aa2-e5c7ec8c0e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:25.495185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262096a-a9c0-4ce8-9aa2-e5c7ec8c0e9f None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:25.495836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262096a-a9c0-4ce8-9aa2-e5c7ec8c0e9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:25.496133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262096a-a9c0-4ce8-9aa2-e5c7ec8c0e9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:25.497599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262096a-a9c0-4ce8-9aa2-e5c7ec8c0e9f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:25.511509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 350/699] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:25.513043 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262096a-a9c0-4ce8-9aa2-e5c7ec8c0e9f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:25.545189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:25.555001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13308f0c-4e3d-4172-9965-174862132291 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:25.593462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65f002-b78e-41db-8998-7daf2848404c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:25.594421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65f002-b78e-41db-8998-7daf2848404c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:25.594957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65f002-b78e-41db-8998-7daf2848404c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:25.595419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65f002-b78e-41db-8998-7daf2848404c None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:25.596450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65f002-b78e-41db-8998-7daf2848404c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:25.597000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65f002-b78e-41db-8998-7daf2848404c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:25.598684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65f002-b78e-41db-8998-7daf2848404c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:25.601587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:25.620799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 350/700] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 17:51:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:25.622469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65f002-b78e-41db-8998-7daf2848404c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:25.646210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262096a-a9c0-4ce8-9aa2-e5c7ec8c0e9f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:25.695930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a6880-73d3-4958-89f6-fdc11de65a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:25.696742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a6880-73d3-4958-89f6-fdc11de65a88 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:25.697020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a6880-73d3-4958-89f6-fdc11de65a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:25.697290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a6880-73d3-4958-89f6-fdc11de65a88 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:25.698766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a6880-73d3-4958-89f6-fdc11de65a88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:25.699073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a6880-73d3-4958-89f6-fdc11de65a88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:25.701075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a6880-73d3-4958-89f6-fdc11de65a88 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:25.716296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 351/701] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:25.717161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a6880-73d3-4958-89f6-fdc11de65a88 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:25.730635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65f002-b78e-41db-8998-7daf2848404c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:25.769416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd9724-a295-4412-9a11-855d564c7416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:25.770022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd9724-a295-4412-9a11-855d564c7416 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:25.770297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd9724-a295-4412-9a11-855d564c7416 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:25.770562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd9724-a295-4412-9a11-855d564c7416 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:25.771195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd9724-a295-4412-9a11-855d564c7416 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:25.772786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd9724-a295-4412-9a11-855d564c7416 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:25.821550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:25.841873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a6880-73d3-4958-89f6-fdc11de65a88 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:25.849136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 351/702] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:51:25 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:25.849960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd9724-a295-4412-9a11-855d564c7416 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:25.921401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:25.945962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fc2a4-9815-4108-a43b-d15f924fced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:25.946771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fc2a4-9815-4108-a43b-d15f924fced7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:25.947148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fc2a4-9815-4108-a43b-d15f924fced7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:25.947568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fc2a4-9815-4108-a43b-d15f924fced7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:25.990175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd9724-a295-4412-9a11-855d564c7416 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:26.029007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec8478-6aa7-4557-ad5f-cd2f6fb29c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:26.029860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec8478-6aa7-4557-ad5f-cd2f6fb29c42 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.030282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec8478-6aa7-4557-ad5f-cd2f6fb29c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.030417 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec8478-6aa7-4557-ad5f-cd2f6fb29c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.031138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec8478-6aa7-4557-ad5f-cd2f6fb29c42 None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=3375c3134ebe473cb5da29f083321cdb) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:26.033122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec8478-6aa7-4557-ad5f-cd2f6fb29c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:26.033862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec8478-6aa7-4557-ad5f-cd2f6fb29c42 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:26.041475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 352/703] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:25 2022] GET /identity/v3/role_assignments?user.id=3375c3134ebe473cb5da29f083321cdb&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:26.075172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3591c-9020-4163-a2f3-9a427aabb88b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.075520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3591c-9020-4163-a2f3-9a427aabb88b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.075821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3591c-9020-4163-a2f3-9a427aabb88b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.077490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 353/704] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:26.083762 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d34b9a-c1c5-4496-9b9d-e6f3f684552e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.084035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d34b9a-c1c5-4496-9b9d-e6f3f684552e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.084307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d34b9a-c1c5-4496-9b9d-e6f3f684552e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.089906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fc2a4-9815-4108-a43b-d15f924fced7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:26.091760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fc2a4-9815-4108-a43b-d15f924fced7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:26.115411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-00d34b9a-c1c5-4496-9b9d-e6f3f684552e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:26.117016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-00d34b9a-c1c5-4496-9b9d-e6f3f684552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:26.188908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:26.216716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 354/705] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:26.223770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 352/706] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:26.230175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d34b9a-c1c5-4496-9b9d-e6f3f684552e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:26.259351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fc2a4-9815-4108-a43b-d15f924fced7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:26.285717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:26.346235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d34b9a-c1c5-4496-9b9d-e6f3f684552e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:26.375831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fc2a4-9815-4108-a43b-d15f924fced7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:26.383718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43d0f4-2418-40ab-9d83-98fa633552a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:26.384613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43d0f4-2418-40ab-9d83-98fa633552a4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.385096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43d0f4-2418-40ab-9d83-98fa633552a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.385536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43d0f4-2418-40ab-9d83-98fa633552a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.397495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43d0f4-2418-40ab-9d83-98fa633552a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:26.399322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43d0f4-2418-40ab-9d83-98fa633552a4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1e43d0f4-2418-40ab-9d83-98fa633552a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:26.415463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:26.416558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:26.417493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:26.417493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:26.417493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:26.418680 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3840bb-b08e-4176-a306-b8ea9d2226cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:26.419495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3840bb-b08e-4176-a306-b8ea9d2226cc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.419666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 355/707] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:26.424090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3840bb-b08e-4176-a306-b8ea9d2226cc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.424090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3840bb-b08e-4176-a306-b8ea9d2226cc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.424090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3840bb-b08e-4176-a306-b8ea9d2226cc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:26.424090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3840bb-b08e-4176-a306-b8ea9d2226cc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:26.425455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43d0f4-2418-40ab-9d83-98fa633552a4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-fa3840bb-b08e-4176-a306-b8ea9d2226cc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:26.437815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:26.438927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:26.439908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 353/708] 10.222.0.50 () {56 vars in 1129 bytes} [Mon Aug 8 17:51:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:26.445867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3840bb-b08e-4176-a306-b8ea9d2226cc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:26.500205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:26.533829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:26.540027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43d0f4-2418-40ab-9d83-98fa633552a4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:26.563306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3840bb-b08e-4176-a306-b8ea9d2226cc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:26.578391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55407f63-5ba3-4597-a63d-b860d4beba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:26.579646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55407f63-5ba3-4597-a63d-b860d4beba63 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.580154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55407f63-5ba3-4597-a63d-b860d4beba63 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.580694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55407f63-5ba3-4597-a63d-b860d4beba63 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.581663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55407f63-5ba3-4597-a63d-b860d4beba63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:26.582209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55407f63-5ba3-4597-a63d-b860d4beba63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:26.583914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55407f63-5ba3-4597-a63d-b860d4beba63 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:26.598605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 356/709] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:26.604268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ed681-058a-4d9b-ae53-8c655e8a3efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:26.604808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55407f63-5ba3-4597-a63d-b860d4beba63 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:26.604918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ed681-058a-4d9b-ae53-8c655e8a3efd None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.605160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ed681-058a-4d9b-ae53-8c655e8a3efd None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.605429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ed681-058a-4d9b-ae53-8c655e8a3efd None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.606103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ed681-058a-4d9b-ae53-8c655e8a3efd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:26.606403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ed681-058a-4d9b-ae53-8c655e8a3efd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:26.608838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ed681-058a-4d9b-ae53-8c655e8a3efd None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:26.616010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 354/710] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 17:51:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:26.622402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ed681-058a-4d9b-ae53-8c655e8a3efd None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:26.728023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55407f63-5ba3-4597-a63d-b860d4beba63 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:26.741515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ed681-058a-4d9b-ae53-8c655e8a3efd None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:26.754616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828001b-9520-4ec8-98f4-932dace8e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:26.754616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828001b-9520-4ec8-98f4-932dace8e630 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.754616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828001b-9520-4ec8-98f4-932dace8e630 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.757297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828001b-9520-4ec8-98f4-932dace8e630 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.759366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:26.785930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828001b-9520-4ec8-98f4-932dace8e630 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:26.788109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828001b-9520-4ec8-98f4-932dace8e630 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c828001b-9520-4ec8-98f4-932dace8e630 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:26.809634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:26.810608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:26.812014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:26.812014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:26.812014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:26.812014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:26.812014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:26.812014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:26.812014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:26.812014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:26.812014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 357/711] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:26.814228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828001b-9520-4ec8-98f4-932dace8e630 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:26.819151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab832cd-a81c-410a-9434-024df3da3dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:26.819968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab832cd-a81c-410a-9434-024df3da3dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.820393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab832cd-a81c-410a-9434-024df3da3dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.820862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab832cd-a81c-410a-9434-024df3da3dc2 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.821649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab832cd-a81c-410a-9434-024df3da3dc2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:26.822502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab832cd-a81c-410a-9434-024df3da3dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:26.824178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab832cd-a81c-410a-9434-024df3da3dc2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:26.834662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:26.838389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 355/712] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 17:51:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:26.924956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828001b-9520-4ec8-98f4-932dace8e630 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:26.959285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e834e5-bbe3-4e73-b777-07bfc19ea757 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.959574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e834e5-bbe3-4e73-b777-07bfc19ea757 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.959788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e834e5-bbe3-4e73-b777-07bfc19ea757 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.961456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 356/713] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:26.963942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be7150-dce9-4554-ab4e-7f893323cec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:26.964627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7150-dce9-4554-ab4e-7f893323cec3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.965086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7150-dce9-4554-ab4e-7f893323cec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.965597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7150-dce9-4554-ab4e-7f893323cec3 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.966460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be7150-dce9-4554-ab4e-7f893323cec3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:26.966760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c9b67-58ff-42f8-bc8f-d460973f4db3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:26.967050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c9b67-58ff-42f8-bc8f-d460973f4db3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:26.967240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be7150-dce9-4554-ab4e-7f893323cec3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:26.967346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c9b67-58ff-42f8-bc8f-d460973f4db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:26.969074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be7150-dce9-4554-ab4e-7f893323cec3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:26.983796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 358/714] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:26.990288 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be7150-dce9-4554-ab4e-7f893323cec3 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:27.021717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-150c9b67-58ff-42f8-bc8f-d460973f4db3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:27.022889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-150c9b67-58ff-42f8-bc8f-d460973f4db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:27.078234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:27.102926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be7150-dce9-4554-ab4e-7f893323cec3 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:27.128112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 357/715] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:51:27.140308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad9984-f2fe-4f9b-a0d9-84d64194c2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:27.140941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad9984-f2fe-4f9b-a0d9-84d64194c2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:27.141205 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad9984-f2fe-4f9b-a0d9-84d64194c2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:27.141467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad9984-f2fe-4f9b-a0d9-84d64194c2c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:27.142430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad9984-f2fe-4f9b-a0d9-84d64194c2c2 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=c496da229db34f90a4b3fdcfa582dca8) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:27.142537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad9984-f2fe-4f9b-a0d9-84d64194c2c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:27.144027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad9984-f2fe-4f9b-a0d9-84d64194c2c2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:27.150481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 359/716] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:26 2022] GET /identity/v3/role_assignments?user.id=c496da229db34f90a4b3fdcfa582dca8&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:27.151170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c9b67-58ff-42f8-bc8f-d460973f4db3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:27.276177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c9b67-58ff-42f8-bc8f-d460973f4db3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:27.279464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:27.320164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73a95d-bae3-42b5-9c8f-049ee596f424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:27.320862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73a95d-bae3-42b5-9c8f-049ee596f424 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:27.321141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73a95d-bae3-42b5-9c8f-049ee596f424 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:27.321406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73a95d-bae3-42b5-9c8f-049ee596f424 None admin] PATH_INFO: `/v3/services/network` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:27.321987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73a95d-bae3-42b5-9c8f-049ee596f424 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:27.323351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73a95d-bae3-42b5-9c8f-049ee596f424 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6f73a95d-bae3-42b5-9c8f-049ee596f424 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:27.334192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:27.335232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 17:51:27.336119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:27.339079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 358/717] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:27 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:27.345417 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad9984-f2fe-4f9b-a0d9-84d64194c2c2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:27.451139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad9984-f2fe-4f9b-a0d9-84d64194c2c2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:27.486585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603d896-d8fe-463d-932a-30d238ad0126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:27.487589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603d896-d8fe-463d-932a-30d238ad0126 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:27.488109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603d896-d8fe-463d-932a-30d238ad0126 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:27.488597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603d896-d8fe-463d-932a-30d238ad0126 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:27.489435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603d896-d8fe-463d-932a-30d238ad0126 None admin] RBAC: Adding query filter params (name=network) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:27.489932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603d896-d8fe-463d-932a-30d238ad0126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:27.491565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603d896-d8fe-463d-932a-30d238ad0126 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:27.496594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 360/718] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:27.501956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73a95d-bae3-42b5-9c8f-049ee596f424 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:27.643827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73a95d-bae3-42b5-9c8f-049ee596f424 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:27.687983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1790baf-579f-4ab3-814b-c3daaf27e77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:27.688616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1790baf-579f-4ab3-814b-c3daaf27e77c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:27.688889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1790baf-579f-4ab3-814b-c3daaf27e77c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:27.689155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1790baf-579f-4ab3-814b-c3daaf27e77c None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:27.689784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1790baf-579f-4ab3-814b-c3daaf27e77c None admin] RBAC: Adding query filter params (type=network) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:27.690078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1790baf-579f-4ab3-814b-c3daaf27e77c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:27.691446 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1790baf-579f-4ab3-814b-c3daaf27e77c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:27.700781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 359/719] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:28.310676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23b0eb-4156-4fd2-8db3-5041e14b127e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:28.311719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23b0eb-4156-4fd2-8db3-5041e14b127e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:28.312206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23b0eb-4156-4fd2-8db3-5041e14b127e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:28.314007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 361/720] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:28.318879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07738e5f-f776-46f0-9f54-a92069978aab None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:28.319290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07738e5f-f776-46f0-9f54-a92069978aab None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:28.319476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07738e5f-f776-46f0-9f54-a92069978aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:28.347513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-07738e5f-f776-46f0-9f54-a92069978aab None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:28.348730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-07738e5f-f776-46f0-9f54-a92069978aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:28.386020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:28.447737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 360/721] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:28.482310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23b0eb-4156-4fd2-8db3-5041e14b127e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:28.492658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:28.618807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23b0eb-4156-4fd2-8db3-5041e14b127e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:28.668948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b7db7-432d-4cc1-a854-f78a043b4a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:28.669575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b7db7-432d-4cc1-a854-f78a043b4a7f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:28.669845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b7db7-432d-4cc1-a854-f78a043b4a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:28.670106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b7db7-432d-4cc1-a854-f78a043b4a7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:28.682944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b7db7-432d-4cc1-a854-f78a043b4a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f36b7db7-432d-4cc1-a854-f78a043b4a7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:28.687995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:28.689032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:28.689934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:28.690655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 362/722] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:28.694890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07738e5f-f776-46f0-9f54-a92069978aab None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:28.867569 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07738e5f-f776-46f0-9f54-a92069978aab None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:28.914061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193d815-028d-4c88-8e2d-0ca11d9ec72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:28.914599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d815-028d-4c88-8e2d-0ca11d9ec72a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:28.914877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d815-028d-4c88-8e2d-0ca11d9ec72a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:28.915146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d815-028d-4c88-8e2d-0ca11d9ec72a None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:28.915810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193d815-028d-4c88-8e2d-0ca11d9ec72a None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:28.916103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193d815-028d-4c88-8e2d-0ca11d9ec72a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:28.917505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193d815-028d-4c88-8e2d-0ca11d9ec72a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:28.930030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 361/723] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:28.935246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b7db7-432d-4cc1-a854-f78a043b4a7f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:29.051647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:29.104186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b7db7-432d-4cc1-a854-f78a043b4a7f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:29.149970 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c5f68-2697-47e4-9954-ad6d7a562149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:29.150592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c5f68-2697-47e4-9954-ad6d7a562149 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.150862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c5f68-2697-47e4-9954-ad6d7a562149 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.151124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c5f68-2697-47e4-9954-ad6d7a562149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.159350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33fac4-215b-42ab-b0fa-43098802a5a8 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.159631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33fac4-215b-42ab-b0fa-43098802a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.159905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33fac4-215b-42ab-b0fa-43098802a5a8 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.161506 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 362/724] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:29.166390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667d610-b6e5-4af3-87a5-e504ecc0dc8e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.166508 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c5f68-2697-47e4-9954-ad6d7a562149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:29.166608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667d610-b6e5-4af3-87a5-e504ecc0dc8e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.166860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667d610-b6e5-4af3-87a5-e504ecc0dc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.167794 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c5f68-2697-47e4-9954-ad6d7a562149 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-232c5f68-2697-47e4-9954-ad6d7a562149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:29.184077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:29.185198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:29.186103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:29.186103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:29.186103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:29.190034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 363/725] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:29.195702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c5f68-2697-47e4-9954-ad6d7a562149 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:29.196735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6667d610-b6e5-4af3-87a5-e504ecc0dc8e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:29.197898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6667d610-b6e5-4af3-87a5-e504ecc0dc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:29.238481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:29.327833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 363/726] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 17:51:29.352958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c5f68-2697-47e4-9954-ad6d7a562149 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:29.369883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667d610-b6e5-4af3-87a5-e504ecc0dc8e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:29.372742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:29.410587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f9fe2-e0fc-43b6-bdf7-e73cc3091fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:29.411191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f9fe2-e0fc-43b6-bdf7-e73cc3091fdd None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.411461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f9fe2-e0fc-43b6-bdf7-e73cc3091fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.411721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f9fe2-e0fc-43b6-bdf7-e73cc3091fdd None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.412358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f9fe2-e0fc-43b6-bdf7-e73cc3091fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:29.413378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f9fe2-e0fc-43b6-bdf7-e73cc3091fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:29.414758 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f9fe2-e0fc-43b6-bdf7-e73cc3091fdd None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:29.438998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 364/727] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:29.443781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f9fe2-e0fc-43b6-bdf7-e73cc3091fdd None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:29.531336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667d610-b6e5-4af3-87a5-e504ecc0dc8e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:29.569226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f9fe2-e0fc-43b6-bdf7-e73cc3091fdd None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:29.585326 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13adaa-fe48-485d-89d7-991a2c9a2d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:29.586598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13adaa-fe48-485d-89d7-991a2c9a2d2f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.587091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13adaa-fe48-485d-89d7-991a2c9a2d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.587756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13adaa-fe48-485d-89d7-991a2c9a2d2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.603358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13adaa-fe48-485d-89d7-991a2c9a2d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:29.605748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a85154-5ee7-41b4-b944-809dd2390c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:29.610901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a85154-5ee7-41b4-b944-809dd2390c8b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.610901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a85154-5ee7-41b4-b944-809dd2390c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.610901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a85154-5ee7-41b4-b944-809dd2390c8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4e13adaa-fe48-485d-89d7-991a2c9a2d2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:29.611145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:29.612017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:29.612898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:29.614895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 364/728] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:29.615637 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a85154-5ee7-41b4-b944-809dd2390c8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:29.616459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fa520-9022-49a6-94cc-bbe57b85e77b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.616914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fa520-9022-49a6-94cc-bbe57b85e77b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.617332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fa520-9022-49a6-94cc-bbe57b85e77b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.617519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a85154-5ee7-41b4-b944-809dd2390c8b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:29.619463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 365/729] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:29.624272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fa520-9022-49a6-94cc-bbe57b85e77b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-38a85154-5ee7-41b4-b944-809dd2390c8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:29.636002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:29.637611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:29.638618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 365/730] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:51:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:29.639253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0771bd3-1a10-44f1-9b2a-ddf182ca3601 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.641061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0771bd3-1a10-44f1-9b2a-ddf182ca3601 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.641061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0771bd3-1a10-44f1-9b2a-ddf182ca3601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.646134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:29.660655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f0771bd3-1a10-44f1-9b2a-ddf182ca3601 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:29.664962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f0771bd3-1a10-44f1-9b2a-ddf182ca3601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:29.693169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:29.734906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 366/731] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:29.736896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b8bcb-10b6-4b70-a836-9b0f85fa55a9 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.737178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b8bcb-10b6-4b70-a836-9b0f85fa55a9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.737446 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b8bcb-10b6-4b70-a836-9b0f85fa55a9 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.742826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 367/732] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:29.742826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b8bcb-10b6-4b70-a836-9b0f85fa55a9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:29.746237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fa520-9022-49a6-94cc-bbe57b85e77b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:29.785759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ee81f-6a3b-489b-8e19-97fdf5ccf5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:29.786355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ee81f-6a3b-489b-8e19-97fdf5ccf5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.786626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ee81f-6a3b-489b-8e19-97fdf5ccf5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.786892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ee81f-6a3b-489b-8e19-97fdf5ccf5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.787552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ee81f-6a3b-489b-8e19-97fdf5ccf5f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:29.787850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ee81f-6a3b-489b-8e19-97fdf5ccf5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:29.789343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ee81f-6a3b-489b-8e19-97fdf5ccf5f4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:29.799721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 366/733] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:29.799721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e35b1a-ccd8-4fbe-80f5-cec6c3e4b79a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.799957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e35b1a-ccd8-4fbe-80f5-cec6c3e4b79a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.800288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e35b1a-ccd8-4fbe-80f5-cec6c3e4b79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.821635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b9e35b1a-ccd8-4fbe-80f5-cec6c3e4b79a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:29.822826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b9e35b1a-ccd8-4fbe-80f5-cec6c3e4b79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:29.864196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b8bcb-10b6-4b70-a836-9b0f85fa55a9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:29.902793 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067e1d-d60d-4046-acba-e93817fe29a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:29.903386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067e1d-d60d-4046-acba-e93817fe29a7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:29.903657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067e1d-d60d-4046-acba-e93817fe29a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:29.903914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067e1d-d60d-4046-acba-e93817fe29a7 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:29.904789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067e1d-d60d-4046-acba-e93817fe29a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:29.905172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067e1d-d60d-4046-acba-e93817fe29a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:29.908568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 367/734] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:29.908740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067e1d-d60d-4046-acba-e93817fe29a7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:29.908823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e35b1a-ccd8-4fbe-80f5-cec6c3e4b79a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:29.923833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:29.937800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 368/735] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 17:51:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:29.938879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067e1d-d60d-4046-acba-e93817fe29a7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:29.965677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:30.023714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e35b1a-ccd8-4fbe-80f5-cec6c3e4b79a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:30.049384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067e1d-d60d-4046-acba-e93817fe29a7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:30.069648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db84dc-3000-45ec-9712-3a992c27b7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:30.070288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db84dc-3000-45ec-9712-3a992c27b7ce None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.070559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db84dc-3000-45ec-9712-3a992c27b7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.070817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db84dc-3000-45ec-9712-3a992c27b7ce None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.071453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db84dc-3000-45ec-9712-3a992c27b7ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:30.072986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db84dc-3000-45ec-9712-3a992c27b7ce None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:30.086828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13aa3a9-adf7-41ad-982c-4ba2fea6022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:30.093950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13aa3a9-adf7-41ad-982c-4ba2fea6022c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.093950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13aa3a9-adf7-41ad-982c-4ba2fea6022c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.093950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13aa3a9-adf7-41ad-982c-4ba2fea6022c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.098600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13aa3a9-adf7-41ad-982c-4ba2fea6022c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:30.100075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13aa3a9-adf7-41ad-982c-4ba2fea6022c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d13aa3a9-adf7-41ad-982c-4ba2fea6022c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:30.114889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:30.116014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:30.117512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:30.117512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:30.117512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:30.117512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 369/736] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:30.117512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13aa3a9-adf7-41ad-982c-4ba2fea6022c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:30.164069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 368/737] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:51:29 2022] POST /identity/v3/services => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:30.164750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db84dc-3000-45ec-9712-3a992c27b7ce None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:30.219441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:30.256270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13aa3a9-adf7-41ad-982c-4ba2fea6022c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:30.276438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:30.325815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f658de-036e-4250-a99f-b7f5206ed28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:30.327072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f658de-036e-4250-a99f-b7f5206ed28c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.327510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f658de-036e-4250-a99f-b7f5206ed28c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.327946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f658de-036e-4250-a99f-b7f5206ed28c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.332929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db84dc-3000-45ec-9712-3a992c27b7ce None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:30.344703 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f658de-036e-4250-a99f-b7f5206ed28c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b3f658de-036e-4250-a99f-b7f5206ed28c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:30.359656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:30.361203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:30.362296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 370/738] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:30.363015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f658de-036e-4250-a99f-b7f5206ed28c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:30.393797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d54c44-ede2-40db-9f35-678154569d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:30.394562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d54c44-ede2-40db-9f35-678154569d48 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.394904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d54c44-ede2-40db-9f35-678154569d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.395172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d54c44-ede2-40db-9f35-678154569d48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.482180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d54c44-ede2-40db-9f35-678154569d48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:30.483660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d54c44-ede2-40db-9f35-678154569d48 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:30.492584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f658de-036e-4250-a99f-b7f5206ed28c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:30.526117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:30.533468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f9ade-94c7-4762-a75b-be2422daa412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:30.534061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f9ade-94c7-4762-a75b-be2422daa412 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.534334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f9ade-94c7-4762-a75b-be2422daa412 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.534598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f9ade-94c7-4762-a75b-be2422daa412 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.535234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f9ade-94c7-4762-a75b-be2422daa412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:30.535529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f9ade-94c7-4762-a75b-be2422daa412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:30.536926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f9ade-94c7-4762-a75b-be2422daa412 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:30.548691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 371/739] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:30.549525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f9ade-94c7-4762-a75b-be2422daa412 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:30.557155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:30.599019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 369/740] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 17:51:30.600583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb1310-758f-410c-b4ef-1fcb84a12db0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.600865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb1310-758f-410c-b4ef-1fcb84a12db0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.601128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb1310-758f-410c-b4ef-1fcb84a12db0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.602585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 370/741] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:30.603128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb1310-758f-410c-b4ef-1fcb84a12db0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:30.698973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f9ade-94c7-4762-a75b-be2422daa412 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:30.708567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb1310-758f-410c-b4ef-1fcb84a12db0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:30.735208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3058035-a810-4af3-90ba-84ad7c83c45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:30.735736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3058035-a810-4af3-90ba-84ad7c83c45c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.736013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3058035-a810-4af3-90ba-84ad7c83c45c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.736302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3058035-a810-4af3-90ba-84ad7c83c45c None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.736959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3058035-a810-4af3-90ba-84ad7c83c45c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:30.737255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3058035-a810-4af3-90ba-84ad7c83c45c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:30.738643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3058035-a810-4af3-90ba-84ad7c83c45c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:30.744371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dfca1-1c84-420f-80bf-8a6fdc9f837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:30.745014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dfca1-1c84-420f-80bf-8a6fdc9f837c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.745282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dfca1-1c84-420f-80bf-8a6fdc9f837c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.745541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dfca1-1c84-420f-80bf-8a6fdc9f837c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.751363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 372/742] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:30.751363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3058035-a810-4af3-90ba-84ad7c83c45c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:30.758557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dfca1-1c84-420f-80bf-8a6fdc9f837c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:30.760109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dfca1-1c84-420f-80bf-8a6fdc9f837c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-461dfca1-1c84-420f-80bf-8a6fdc9f837c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:30.772746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:30.773712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:30.774691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 371/743] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:30.775288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dc6a5-7f31-4018-961b-26d41001ae06 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.775288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dc6a5-7f31-4018-961b-26d41001ae06 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.775924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dc6a5-7f31-4018-961b-26d41001ae06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.800771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a91dc6a5-7f31-4018-961b-26d41001ae06 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:30.801948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a91dc6a5-7f31-4018-961b-26d41001ae06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:30.816839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:30.825335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:30.864368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3058035-a810-4af3-90ba-84ad7c83c45c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:30.884550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 372/744] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 17:51:30.884550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91dc6a5-7f31-4018-961b-26d41001ae06 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:30.925637 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dacf4-ee68-4167-b33a-b4687066ba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:30.925637 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dacf4-ee68-4167-b33a-b4687066ba08 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:30.925637 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dacf4-ee68-4167-b33a-b4687066ba08 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:30.925637 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dacf4-ee68-4167-b33a-b4687066ba08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:30.940890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dacf4-ee68-4167-b33a-b4687066ba08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:30.942415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dacf4-ee68-4167-b33a-b4687066ba08 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b02dacf4-ee68-4167-b33a-b4687066ba08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:30.957064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:30.958047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:30.958933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:30.958933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:30.958933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:30.958933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 373/745] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:30.959599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dacf4-ee68-4167-b33a-b4687066ba08 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.000492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91dc6a5-7f31-4018-961b-26d41001ae06 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:31.037573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f2662-8dc7-4e01-9fdb-781dfd1dac20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:31.038666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f2662-8dc7-4e01-9fdb-781dfd1dac20 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:31.038938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f2662-8dc7-4e01-9fdb-781dfd1dac20 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:31.039352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f2662-8dc7-4e01-9fdb-781dfd1dac20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:31.049991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f2662-8dc7-4e01-9fdb-781dfd1dac20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:31.051497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f2662-8dc7-4e01-9fdb-781dfd1dac20 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-974f2662-8dc7-4e01-9fdb-781dfd1dac20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:31.066115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:31.067322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:31.068526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:31.068526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:31.068526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:31.068526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 373/746] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:31.068526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f2662-8dc7-4e01-9fdb-781dfd1dac20 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.077806 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dacf4-ee68-4167-b33a-b4687066ba08 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:31.079362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:31.085987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:31.116290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c28ee-8d19-44e3-91fe-528e969fa37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:31.117189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c28ee-8d19-44e3-91fe-528e969fa37a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:31.117467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c28ee-8d19-44e3-91fe-528e969fa37a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:31.117734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c28ee-8d19-44e3-91fe-528e969fa37a None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:31.118629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c28ee-8d19-44e3-91fe-528e969fa37a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:31.119228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c28ee-8d19-44e3-91fe-528e969fa37a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:31.120691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c28ee-8d19-44e3-91fe-528e969fa37a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:31.148598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 374/747] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 17:51:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:31.148598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c28ee-8d19-44e3-91fe-528e969fa37a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.174798 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f2662-8dc7-4e01-9fdb-781dfd1dac20 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:31.210845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46461de4-3f95-4c7d-a504-5282693357fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:31.211448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46461de4-3f95-4c7d-a504-5282693357fd None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:31.211799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46461de4-3f95-4c7d-a504-5282693357fd None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:31.212127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46461de4-3f95-4c7d-a504-5282693357fd None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:31.212853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46461de4-3f95-4c7d-a504-5282693357fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:31.214218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46461de4-3f95-4c7d-a504-5282693357fd None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:31.256407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c28ee-8d19-44e3-91fe-528e969fa37a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:31.267084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 374/748] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:51:31 2022] POST /identity/v3/services => generated 240 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:31.268838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46461de4-3f95-4c7d-a504-5282693357fd None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.302493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76daede-4325-4770-b3a9-83fbf8a863ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:31.303163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76daede-4325-4770-b3a9-83fbf8a863ce None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:31.303432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76daede-4325-4770-b3a9-83fbf8a863ce None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:31.303692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76daede-4325-4770-b3a9-83fbf8a863ce None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:31.304390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76daede-4325-4770-b3a9-83fbf8a863ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:31.304712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76daede-4325-4770-b3a9-83fbf8a863ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:31.306211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76daede-4325-4770-b3a9-83fbf8a863ce None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:31.324635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 375/749] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:31.325658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76daede-4325-4770-b3a9-83fbf8a863ce None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.351131 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:31.376130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:31.415640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46461de4-3f95-4c7d-a504-5282693357fd None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:31.458633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9898e-dbb3-435c-a912-99d159c888a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:31.459237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9898e-dbb3-435c-a912-99d159c888a7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:31.459554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9898e-dbb3-435c-a912-99d159c888a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:31.459763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9898e-dbb3-435c-a912-99d159c888a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:31.460397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9898e-dbb3-435c-a912-99d159c888a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:31.460783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9898e-dbb3-435c-a912-99d159c888a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:31.462064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9898e-dbb3-435c-a912-99d159c888a7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:31.472337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76daede-4325-4770-b3a9-83fbf8a863ce None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:31.476016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 375/750] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:31.476913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9898e-dbb3-435c-a912-99d159c888a7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.519372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cfe2d-ea16-4f28-92ea-749f1643af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:31.519953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cfe2d-ea16-4f28-92ea-749f1643af07 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:31.520238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cfe2d-ea16-4f28-92ea-749f1643af07 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:31.520600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cfe2d-ea16-4f28-92ea-749f1643af07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:31.599450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9898e-dbb3-435c-a912-99d159c888a7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:31.619208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cfe2d-ea16-4f28-92ea-749f1643af07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:31.621437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cfe2d-ea16-4f28-92ea-749f1643af07 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:31.631499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:31.639535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947b52b-d52d-4bba-bef1-da4c0a6cfcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:31.640393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947b52b-d52d-4bba-bef1-da4c0a6cfcbb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:31.640541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947b52b-d52d-4bba-bef1-da4c0a6cfcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:31.640919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947b52b-d52d-4bba-bef1-da4c0a6cfcbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:31.647955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947b52b-d52d-4bba-bef1-da4c0a6cfcbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:31.649596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947b52b-d52d-4bba-bef1-da4c0a6cfcbb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d947b52b-d52d-4bba-bef1-da4c0a6cfcbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:31.661922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:31.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:31.664091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:31.664091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:31.664091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:31.664091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:31.664091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:31.664091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:31.664091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:31.664091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:31.664920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:31.666585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 376/751] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:31.667613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947b52b-d52d-4bba-bef1-da4c0a6cfcbb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.762813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 376/752] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:51:31.764070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cfe2d-ea16-4f28-92ea-749f1643af07 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.806070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947b52b-d52d-4bba-bef1-da4c0a6cfcbb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:31.851468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86ca8a-89ce-487b-ba78-825c60e83465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:31.852094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86ca8a-89ce-487b-ba78-825c60e83465 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:31.852366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86ca8a-89ce-487b-ba78-825c60e83465 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:31.852660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86ca8a-89ce-487b-ba78-825c60e83465 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:31.863859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86ca8a-89ce-487b-ba78-825c60e83465 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:31.865850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86ca8a-89ce-487b-ba78-825c60e83465 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:31.877071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cfe2d-ea16-4f28-92ea-749f1643af07 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4c86ca8a-89ce-487b-ba78-825c60e83465 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:31.881623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:31.882655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:31.883636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 377/753] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 17:51:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:31.884235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86ca8a-89ce-487b-ba78-825c60e83465 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.904317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:31.918866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1971a3a-a3fa-4890-b572-144dfd6fd60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:31.919511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971a3a-a3fa-4890-b572-144dfd6fd60f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:31.924126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971a3a-a3fa-4890-b572-144dfd6fd60f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:31.924126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971a3a-a3fa-4890-b572-144dfd6fd60f None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:31.924126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1971a3a-a3fa-4890-b572-144dfd6fd60f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:31.924126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1971a3a-a3fa-4890-b572-144dfd6fd60f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:31.924126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1971a3a-a3fa-4890-b572-144dfd6fd60f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:31.939892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 377/754] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:31.939892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1971a3a-a3fa-4890-b572-144dfd6fd60f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:31.954549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:32.034385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86ca8a-89ce-487b-ba78-825c60e83465 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.051097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1971a3a-a3fa-4890-b572-144dfd6fd60f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.080298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce0b55-f860-4f43-808d-e85954c730cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:32.081287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce0b55-f860-4f43-808d-e85954c730cb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.081704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce0b55-f860-4f43-808d-e85954c730cb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.082108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce0b55-f860-4f43-808d-e85954c730cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.088082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a73e3-2ce8-4241-afdc-533e83e230fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:32.088705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a73e3-2ce8-4241-afdc-533e83e230fd None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.088972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a73e3-2ce8-4241-afdc-533e83e230fd None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.089291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a73e3-2ce8-4241-afdc-533e83e230fd None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.090587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a73e3-2ce8-4241-afdc-533e83e230fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:32.090850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a73e3-2ce8-4241-afdc-533e83e230fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:32.092564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a73e3-2ce8-4241-afdc-533e83e230fd None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:32.095724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce0b55-f860-4f43-808d-e85954c730cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:32.097330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce0b55-f860-4f43-808d-e85954c730cb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-9dce0b55-f860-4f43-808d-e85954c730cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:32.112304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:32.113533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:32.114619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:32.114619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:32.114619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:32.115167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 378/755] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 17:51:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:32.116067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a73e3-2ce8-4241-afdc-533e83e230fd None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:32.120953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 378/756] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:32.122260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce0b55-f860-4f43-808d-e85954c730cb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:32.157628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:32.218983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a73e3-2ce8-4241-afdc-533e83e230fd None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.228688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:32.232273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce0b55-f860-4f43-808d-e85954c730cb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.254386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82eed9-ab3e-4693-925d-2eb73b7ac8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:32.255464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82eed9-ab3e-4693-925d-2eb73b7ac8bb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.255918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82eed9-ab3e-4693-925d-2eb73b7ac8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.256356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82eed9-ab3e-4693-925d-2eb73b7ac8bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.257227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82eed9-ab3e-4693-925d-2eb73b7ac8bb None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=4f0ca85ea7974962b8115428563aef8f) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:32.257579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82eed9-ab3e-4693-925d-2eb73b7ac8bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:32.258993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82eed9-ab3e-4693-925d-2eb73b7ac8bb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:32.268536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 379/757] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:32 2022] GET /identity/v3/role_assignments?user.id=4f0ca85ea7974962b8115428563aef8f&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:32.268536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82eed9-ab3e-4693-925d-2eb73b7ac8bb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:32.280485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd581c-518f-47b6-807b-4b089040180b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:32.281422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd581c-518f-47b6-807b-4b089040180b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.281859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd581c-518f-47b6-807b-4b089040180b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.282331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd581c-518f-47b6-807b-4b089040180b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.306000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd581c-518f-47b6-807b-4b089040180b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:32.307727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd581c-518f-47b6-807b-4b089040180b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-57dd581c-518f-47b6-807b-4b089040180b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:32.336460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:32.337666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:32.338769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:32.338769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:32.338769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:32.346084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 379/758] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:32.356854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd581c-518f-47b6-807b-4b089040180b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:32.385953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82eed9-ab3e-4693-925d-2eb73b7ac8bb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.388204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:32.425921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c12ca-9d02-46d3-95fb-424c7f818158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:32.426530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c12ca-9d02-46d3-95fb-424c7f818158 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.426798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c12ca-9d02-46d3-95fb-424c7f818158 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.427061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c12ca-9d02-46d3-95fb-424c7f818158 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.427704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c12ca-9d02-46d3-95fb-424c7f818158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:32.427998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c12ca-9d02-46d3-95fb-424c7f818158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:32.429481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c12ca-9d02-46d3-95fb-424c7f818158 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:32.443309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 380/759] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:32.449144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c12ca-9d02-46d3-95fb-424c7f818158 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:32.576148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c12ca-9d02-46d3-95fb-424c7f818158 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.616393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649c05b-2f62-4e44-9c4e-76ca48b49f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:32.617386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649c05b-2f62-4e44-9c4e-76ca48b49f1a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.617827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649c05b-2f62-4e44-9c4e-76ca48b49f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.618303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649c05b-2f62-4e44-9c4e-76ca48b49f1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.626385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649c05b-2f62-4e44-9c4e-76ca48b49f1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:32.628069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649c05b-2f62-4e44-9c4e-76ca48b49f1a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:32.636097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd581c-518f-47b6-807b-4b089040180b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-a649c05b-2f62-4e44-9c4e-76ca48b49f1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:32.641265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:32.642314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:32.643219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:32.643219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:32.643219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:32.643219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:32.643219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:32.643219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:32.643219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:32.643219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:32.645661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 381/760] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:32.647255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae38cca-e6d7-4fee-baeb-9e53d68b32f4 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.647848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae38cca-e6d7-4fee-baeb-9e53d68b32f4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.648457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae38cca-e6d7-4fee-baeb-9e53d68b32f4 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.649764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 382/761] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:32.651706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae38cca-e6d7-4fee-baeb-9e53d68b32f4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:32.682385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:32.685283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb362ec5-4b54-4e53-a403-37f610499d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:32.686089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb362ec5-4b54-4e53-a403-37f610499d13 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.686804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb362ec5-4b54-4e53-a403-37f610499d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.687469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb362ec5-4b54-4e53-a403-37f610499d13 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.689390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb362ec5-4b54-4e53-a403-37f610499d13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:32.689804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb362ec5-4b54-4e53-a403-37f610499d13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:32.691232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb362ec5-4b54-4e53-a403-37f610499d13 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:32.708578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 380/762] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:32.710405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29758a7-40cf-4aa8-9a9d-6b23c8e0cfbd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.710866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29758a7-40cf-4aa8-9a9d-6b23c8e0cfbd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.711349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29758a7-40cf-4aa8-9a9d-6b23c8e0cfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.731976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e29758a7-40cf-4aa8-9a9d-6b23c8e0cfbd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:32.733899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e29758a7-40cf-4aa8-9a9d-6b23c8e0cfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:32.742875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:32.801317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae38cca-e6d7-4fee-baeb-9e53d68b32f4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.807160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 381/763] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:32.808390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29758a7-40cf-4aa8-9a9d-6b23c8e0cfbd None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:32.848408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22174d86-a491-44ea-a5fe-459fed0a4569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:32.849414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174d86-a491-44ea-a5fe-459fed0a4569 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.849693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174d86-a491-44ea-a5fe-459fed0a4569 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.849961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174d86-a491-44ea-a5fe-459fed0a4569 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.850649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22174d86-a491-44ea-a5fe-459fed0a4569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:32.850946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22174d86-a491-44ea-a5fe-459fed0a4569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:32.852791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22174d86-a491-44ea-a5fe-459fed0a4569 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:32.870042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 383/764] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:32.870838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22174d86-a491-44ea-a5fe-459fed0a4569 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:32.919546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29758a7-40cf-4aa8-9a9d-6b23c8e0cfbd None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.957754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd0e81-288b-43ef-8c10-508004e7d758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:32.958921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd0e81-288b-43ef-8c10-508004e7d758 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:32.959442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd0e81-288b-43ef-8c10-508004e7d758 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:32.959929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd0e81-288b-43ef-8c10-508004e7d758 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:32.961751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:32.968737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd0e81-288b-43ef-8c10-508004e7d758 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:32.970472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd0e81-288b-43ef-8c10-508004e7d758 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:32.980120 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22174d86-a491-44ea-a5fe-459fed0a4569 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-2bcd0e81-288b-43ef-8c10-508004e7d758 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:32.983178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:32.988989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 382/765] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:32.989895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd0e81-288b-43ef-8c10-508004e7d758 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:32.991445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:33.018185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b95748-a08d-49e0-87fa-304ba4ba786f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:33.020932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b95748-a08d-49e0-87fa-304ba4ba786f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.020932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b95748-a08d-49e0-87fa-304ba4ba786f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.020932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b95748-a08d-49e0-87fa-304ba4ba786f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.020932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b95748-a08d-49e0-87fa-304ba4ba786f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:33.021232 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b95748-a08d-49e0-87fa-304ba4ba786f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-07b95748-a08d-49e0-87fa-304ba4ba786f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:33.032022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:33.033344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:33.034471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 384/766] 10.222.0.50 () {56 vars in 1129 bytes} [Mon Aug 8 17:51:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:33.035273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b95748-a08d-49e0-87fa-304ba4ba786f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:33.107792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd0e81-288b-43ef-8c10-508004e7d758 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:33.137513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b95748-a08d-49e0-87fa-304ba4ba786f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:33.145344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513382c-adb9-44ce-93b9-6e44d1c796da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:33.146339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513382c-adb9-44ce-93b9-6e44d1c796da None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.146903 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513382c-adb9-44ce-93b9-6e44d1c796da None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.147362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513382c-adb9-44ce-93b9-6e44d1c796da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.148438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513382c-adb9-44ce-93b9-6e44d1c796da None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=3375c3134ebe473cb5da29f083321cdb) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:33.148956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513382c-adb9-44ce-93b9-6e44d1c796da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:33.150602 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513382c-adb9-44ce-93b9-6e44d1c796da None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:33.157933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 383/767] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:32 2022] GET /identity/v3/role_assignments?user.id=3375c3134ebe473cb5da29f083321cdb&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:33.159139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513382c-adb9-44ce-93b9-6e44d1c796da None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:33.181493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce62d6-0d39-49e5-9559-7bdace75ca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:33.182132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce62d6-0d39-49e5-9559-7bdace75ca6b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.182388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce62d6-0d39-49e5-9559-7bdace75ca6b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.182655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce62d6-0d39-49e5-9559-7bdace75ca6b None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.183364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce62d6-0d39-49e5-9559-7bdace75ca6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:33.183702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce62d6-0d39-49e5-9559-7bdace75ca6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:33.186969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce62d6-0d39-49e5-9559-7bdace75ca6b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:33.214531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 385/768] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:33.221030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce62d6-0d39-49e5-9559-7bdace75ca6b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:33.233188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:33.315448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:33.349449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce62d6-0d39-49e5-9559-7bdace75ca6b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:33.382380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513382c-adb9-44ce-93b9-6e44d1c796da None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:33.400603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6c96d-3e81-4406-becb-024da98eb1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:33.401834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c96d-3e81-4406-becb-024da98eb1d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.403091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c96d-3e81-4406-becb-024da98eb1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.403931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c96d-3e81-4406-becb-024da98eb1d4 None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/c496da229db34f90a4b3fdcfa582dca8/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.406146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6c96d-3e81-4406-becb-024da98eb1d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=c496da229db34f90a4b3fdcfa582dca8, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:33.408555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6c96d-3e81-4406-becb-024da98eb1d4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:33.435999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeefbd4-00f3-47b6-9698-6de49b0c836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:33.437108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeefbd4-00f3-47b6-9698-6de49b0c836f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.437586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeefbd4-00f3-47b6-9698-6de49b0c836f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.438149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeefbd4-00f3-47b6-9698-6de49b0c836f None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.439041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeefbd4-00f3-47b6-9698-6de49b0c836f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:33.439527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeefbd4-00f3-47b6-9698-6de49b0c836f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:33.441200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeefbd4-00f3-47b6-9698-6de49b0c836f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:33.447057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 384/769] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 17:51:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:33.452945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeefbd4-00f3-47b6-9698-6de49b0c836f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:33.516545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 386/770] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:51:33 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/c496da229db34f90a4b3fdcfa582dca8/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:51:33.520642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42d72c-6770-4ded-a9a8-b0513a3ff9d8 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.520642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42d72c-6770-4ded-a9a8-b0513a3ff9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.520642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42d72c-6770-4ded-a9a8-b0513a3ff9d8 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.520642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 387/771] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:33.528041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53884af6-dce4-4411-838d-a3cc511aed07 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.529614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53884af6-dce4-4411-838d-a3cc511aed07 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.529961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53884af6-dce4-4411-838d-a3cc511aed07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.564349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-53884af6-dce4-4411-838d-a3cc511aed07 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:33.570874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-53884af6-dce4-4411-838d-a3cc511aed07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:33.634919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeefbd4-00f3-47b6-9698-6de49b0c836f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:33.672783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:33.689628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 388/772] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:33.696375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:33.701411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f43fa-38f6-4d40-a168-7eb5d72f6d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:33.702080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f43fa-38f6-4d40-a168-7eb5d72f6d1b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.702352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f43fa-38f6-4d40-a168-7eb5d72f6d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.702618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f43fa-38f6-4d40-a168-7eb5d72f6d1b None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.703252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f43fa-38f6-4d40-a168-7eb5d72f6d1b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:33.703590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f43fa-38f6-4d40-a168-7eb5d72f6d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:33.705382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f43fa-38f6-4d40-a168-7eb5d72f6d1b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:33.711300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 385/773] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 17:51:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:33.717884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53884af6-dce4-4411-838d-a3cc511aed07 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:33.727276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f43fa-38f6-4d40-a168-7eb5d72f6d1b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:33.880847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53884af6-dce4-4411-838d-a3cc511aed07 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:33.905082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f43fa-38f6-4d40-a168-7eb5d72f6d1b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:33.942631 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b50e-3d59-4172-bfac-66246c6925d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:33.943316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b50e-3d59-4172-bfac-66246c6925d7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.943689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b50e-3d59-4172-bfac-66246c6925d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.944058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b50e-3d59-4172-bfac-66246c6925d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.944899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b50e-3d59-4172-bfac-66246c6925d7 None admin] RBAC: Adding query filter params (interface=public, service_id=6102e209d7d64281ba68d59d1de285df, region_id=RegionOne) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:33.945302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b50e-3d59-4172-bfac-66246c6925d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:33.946798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b50e-3d59-4172-bfac-66246c6925d7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:33.965396 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 389/774] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 17:51:33 2022] GET /identity/v3/endpoints?service_id=6102e209d7d64281ba68d59d1de285df&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:33.966947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b50e-3d59-4172-bfac-66246c6925d7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:33.974086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd83398-8e00-4ff4-a733-0770ab7b9b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:33.974838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd83398-8e00-4ff4-a733-0770ab7b9b55 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:33.975213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd83398-8e00-4ff4-a733-0770ab7b9b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:33.975579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd83398-8e00-4ff4-a733-0770ab7b9b55 None admin] PATH_INFO: `/v3/services/network` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:33.976263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd83398-8e00-4ff4-a733-0770ab7b9b55 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:33.977769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd83398-8e00-4ff4-a733-0770ab7b9b55 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6fd83398-8e00-4ff4-a733-0770ab7b9b55 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:33.994157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:33.995395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 17:51:33.996968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:34.001577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 386/775] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:33 2022] GET /identity/v3/services/network => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:34.006228 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd83398-8e00-4ff4-a733-0770ab7b9b55 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:34.093024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:34.099380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:34.170378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b50e-3d59-4172-bfac-66246c6925d7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:34.181896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd83398-8e00-4ff4-a733-0770ab7b9b55 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:34.240484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3001d-fa9d-490b-918c-27d8d38c1749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:34.241215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3001d-fa9d-490b-918c-27d8d38c1749 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:34.241591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3001d-fa9d-490b-918c-27d8d38c1749 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:34.241957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3001d-fa9d-490b-918c-27d8d38c1749 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:34.242664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3001d-fa9d-490b-918c-27d8d38c1749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:34.244129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3001d-fa9d-490b-918c-27d8d38c1749 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:34.246861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2677c6-ec9c-4b90-b42c-e7da582fcce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:34.247565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2677c6-ec9c-4b90-b42c-e7da582fcce8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:34.248610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2677c6-ec9c-4b90-b42c-e7da582fcce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:34.249927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2677c6-ec9c-4b90-b42c-e7da582fcce8 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:34.250678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2677c6-ec9c-4b90-b42c-e7da582fcce8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:34.251243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2677c6-ec9c-4b90-b42c-e7da582fcce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:34.254887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2677c6-ec9c-4b90-b42c-e7da582fcce8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:34.260627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 390/776] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 17:51:33 2022] GET /identity/v3/services => generated 772 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:34.268315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 387/777] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:34.273912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3001d-fa9d-490b-918c-27d8d38c1749 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:34.388641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3001d-fa9d-490b-918c-27d8d38c1749 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:34.421335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:34.432756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d813f-ec0b-4706-9b89-11456c3d18bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:34.433420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d813f-ec0b-4706-9b89-11456c3d18bf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:34.434315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d813f-ec0b-4706-9b89-11456c3d18bf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:34.434506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d813f-ec0b-4706-9b89-11456c3d18bf None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:34.435125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d813f-ec0b-4706-9b89-11456c3d18bf None admin] RBAC: Adding query filter params (type=network) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:34.435426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d813f-ec0b-4706-9b89-11456c3d18bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:34.436817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d813f-ec0b-4706-9b89-11456c3d18bf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:34.443235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 391/778] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:34.449718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2677c6-ec9c-4b90-b42c-e7da582fcce8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:34.560761 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2677c6-ec9c-4b90-b42c-e7da582fcce8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:34.588672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:34.591676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2c2a7-4df9-4d1e-9f68-16e3a233678b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:34.591955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2c2a7-4df9-4d1e-9f68-16e3a233678b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:34.592222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2c2a7-4df9-4d1e-9f68-16e3a233678b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:34.593967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 392/779] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:34.598928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a50f83-31d4-42e4-a0c4-4b579c21f279 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:34.599203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a50f83-31d4-42e4-a0c4-4b579c21f279 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:34.599466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a50f83-31d4-42e4-a0c4-4b579c21f279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:34.604912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cc134-2232-426b-baf4-dbff2f9e91da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:34.611638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cc134-2232-426b-baf4-dbff2f9e91da None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:34.611638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cc134-2232-426b-baf4-dbff2f9e91da None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:34.611638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cc134-2232-426b-baf4-dbff2f9e91da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:34.611638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cc134-2232-426b-baf4-dbff2f9e91da None admin] RBAC: Adding query filter params (interface=public, service_id=598cd843729c4af59cf6cde4d96ef294, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:34.611638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cc134-2232-426b-baf4-dbff2f9e91da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:34.611638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cc134-2232-426b-baf4-dbff2f9e91da None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:34.618777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 388/780] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 17:51:34 2022] GET /identity/v3/endpoints?service_id=598cd843729c4af59cf6cde4d96ef294&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:34.625990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d6a50f83-31d4-42e4-a0c4-4b579c21f279 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:34.627362 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d6a50f83-31d4-42e4-a0c4-4b579c21f279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:34.630853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cc134-2232-426b-baf4-dbff2f9e91da None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:34.739706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 393/781] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:51:34.784036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a50f83-31d4-42e4-a0c4-4b579c21f279 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:34.807213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cc134-2232-426b-baf4-dbff2f9e91da None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:34.870648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de802679-cfb2-4ecf-8010-65ea92660257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:34.871407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de802679-cfb2-4ecf-8010-65ea92660257 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:34.871519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de802679-cfb2-4ecf-8010-65ea92660257 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:34.871703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de802679-cfb2-4ecf-8010-65ea92660257 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:34.872296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de802679-cfb2-4ecf-8010-65ea92660257 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:34.873662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de802679-cfb2-4ecf-8010-65ea92660257 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:34.883749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 389/782] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 17:51:34 2022] GET /identity/v3/services => generated 772 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:34.962604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a50f83-31d4-42e4-a0c4-4b579c21f279 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:34.994426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:35.020364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e534e-e66e-473d-bdad-e4f5ca0ee479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:35.021038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e534e-e66e-473d-bdad-e4f5ca0ee479 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:35.021312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e534e-e66e-473d-bdad-e4f5ca0ee479 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:35.021578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e534e-e66e-473d-bdad-e4f5ca0ee479 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:35.041094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e534e-e66e-473d-bdad-e4f5ca0ee479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-0e1e534e-e66e-473d-bdad-e4f5ca0ee479 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:35.046102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:35.047086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:35.047960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:35.051906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 394/783] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:35.053215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de802679-cfb2-4ecf-8010-65ea92660257 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:35.134126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:35.186040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de802679-cfb2-4ecf-8010-65ea92660257 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:35.227104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ec600-16d7-44c5-9fb2-f6f10cb05990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:35.227706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ec600-16d7-44c5-9fb2-f6f10cb05990 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:35.228033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ec600-16d7-44c5-9fb2-f6f10cb05990 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:35.228297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ec600-16d7-44c5-9fb2-f6f10cb05990 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:35.229129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ec600-16d7-44c5-9fb2-f6f10cb05990 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:35.229430 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ec600-16d7-44c5-9fb2-f6f10cb05990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:35.230892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ec600-16d7-44c5-9fb2-f6f10cb05990 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:35.241613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 390/784] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:35.246874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e534e-e66e-473d-bdad-e4f5ca0ee479 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:35.381197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e534e-e66e-473d-bdad-e4f5ca0ee479 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:35.432084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff4b41-b602-4c4d-a83c-0d1035244299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:35.432648 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff4b41-b602-4c4d-a83c-0d1035244299 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:35.432921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff4b41-b602-4c4d-a83c-0d1035244299 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:35.433184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff4b41-b602-4c4d-a83c-0d1035244299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:35.452778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff4b41-b602-4c4d-a83c-0d1035244299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:35.454511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff4b41-b602-4c4d-a83c-0d1035244299 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-15ff4b41-b602-4c4d-a83c-0d1035244299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:35.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:35.474028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:35.474974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:35.474974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:35.474974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:35.474974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 395/785] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:35.479742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ec600-16d7-44c5-9fb2-f6f10cb05990 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:35.562512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5781-82d4-4ee5-864e-d74cf41b4e59 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:35.562841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5781-82d4-4ee5-864e-d74cf41b4e59 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:35.563056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5781-82d4-4ee5-864e-d74cf41b4e59 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:35.565541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 396/786] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:35.573798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f57443-c5df-42db-9916-55e3b1b2f882 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:35.574079 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f57443-c5df-42db-9916-55e3b1b2f882 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:35.574438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f57443-c5df-42db-9916-55e3b1b2f882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:35.613633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-98f57443-c5df-42db-9916-55e3b1b2f882 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:35.614891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-98f57443-c5df-42db-9916-55e3b1b2f882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:35.633505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ec600-16d7-44c5-9fb2-f6f10cb05990 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:35.656749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:35.664923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:35.695254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cdde8-8f3b-4a62-9654-0c8b7f3f1e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:35.695792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cdde8-8f3b-4a62-9654-0c8b7f3f1e1b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:35.696070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cdde8-8f3b-4a62-9654-0c8b7f3f1e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:35.696338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cdde8-8f3b-4a62-9654-0c8b7f3f1e1b None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:35.697013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170cdde8-8f3b-4a62-9654-0c8b7f3f1e1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:35.697314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170cdde8-8f3b-4a62-9654-0c8b7f3f1e1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:35.700056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170cdde8-8f3b-4a62-9654-0c8b7f3f1e1b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:35.717085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 391/787] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:35.723264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cdde8-8f3b-4a62-9654-0c8b7f3f1e1b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:35.759126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 397/788] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:35.793253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f57443-c5df-42db-9916-55e3b1b2f882 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:35.837733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cdde8-8f3b-4a62-9654-0c8b7f3f1e1b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:35.877143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b8111-0f91-4df7-831d-958ead7e9521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:35.878155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8111-0f91-4df7-831d-958ead7e9521 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:35.878677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8111-0f91-4df7-831d-958ead7e9521 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:35.879155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8111-0f91-4df7-831d-958ead7e9521 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:35.887821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b8111-0f91-4df7-831d-958ead7e9521 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:35.889576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b8111-0f91-4df7-831d-958ead7e9521 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-087b8111-0f91-4df7-831d-958ead7e9521 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:35.901808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:35.903517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:35.904462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:35.907273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 392/789] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:51:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:35.912345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b8111-0f91-4df7-831d-958ead7e9521 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:35.948267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:36.019740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f57443-c5df-42db-9916-55e3b1b2f882 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.039431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b8111-0f91-4df7-831d-958ead7e9521 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.056087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e4285-c331-4cc9-ad21-c6a069b2c788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.056816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e4285-c331-4cc9-ad21-c6a069b2c788 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.057089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e4285-c331-4cc9-ad21-c6a069b2c788 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.057350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e4285-c331-4cc9-ad21-c6a069b2c788 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.067957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e4285-c331-4cc9-ad21-c6a069b2c788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-cb1e4285-c331-4cc9-ad21-c6a069b2c788 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:36.073025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:36.074046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:36.075400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:36.079769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 398/790] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:36.081050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccfcf6-73aa-49b4-b72d-7fc3286eadf0 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.081324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccfcf6-73aa-49b4-b72d-7fc3286eadf0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.081589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccfcf6-73aa-49b4-b72d-7fc3286eadf0 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.083086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 399/791] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:36.085424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d569fa-4659-4496-a0ca-355b90c20394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.086264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d569fa-4659-4496-a0ca-355b90c20394 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.086734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d569fa-4659-4496-a0ca-355b90c20394 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.087193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d569fa-4659-4496-a0ca-355b90c20394 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.088100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d569fa-4659-4496-a0ca-355b90c20394 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:36.088205 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7a20b-aae9-41eb-8e10-4f6db366d25d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.088543 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7a20b-aae9-41eb-8e10-4f6db366d25d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.088809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7a20b-aae9-41eb-8e10-4f6db366d25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.089138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d569fa-4659-4496-a0ca-355b90c20394 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:36.091339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d569fa-4659-4496-a0ca-355b90c20394 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:36.108645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-eef7a20b-aae9-41eb-8e10-4f6db366d25d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:36.109548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-eef7a20b-aae9-41eb-8e10-4f6db366d25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:36.113632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 393/792] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 17:51:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:36.114723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d569fa-4659-4496-a0ca-355b90c20394 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:36.129934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:36.194445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 400/793] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:36.195602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7a20b-aae9-41eb-8e10-4f6db366d25d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:36.233554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:36.243068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d569fa-4659-4496-a0ca-355b90c20394 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.280003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d17dc-6937-4332-866e-c72a7c83c64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.280621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d17dc-6937-4332-866e-c72a7c83c64e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.280893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d17dc-6937-4332-866e-c72a7c83c64e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.281150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d17dc-6937-4332-866e-c72a7c83c64e None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.281797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d17dc-6937-4332-866e-c72a7c83c64e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:36.282090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d17dc-6937-4332-866e-c72a7c83c64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:36.283506 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d17dc-6937-4332-866e-c72a7c83c64e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:36.296618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 394/794] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:36.297476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d17dc-6937-4332-866e-c72a7c83c64e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:36.308066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7a20b-aae9-41eb-8e10-4f6db366d25d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.347692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceb4a4-144c-4933-bbae-49478f32d570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.349234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceb4a4-144c-4933-bbae-49478f32d570 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.350107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceb4a4-144c-4933-bbae-49478f32d570 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.350843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceb4a4-144c-4933-bbae-49478f32d570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.364301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceb4a4-144c-4933-bbae-49478f32d570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:36.366127 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceb4a4-144c-4933-bbae-49478f32d570 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-46ceb4a4-144c-4933-bbae-49478f32d570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:36.383505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:36.384555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:36.385455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:36.385455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:36.385455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:36.388324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 401/795] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:36.389502 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceb4a4-144c-4933-bbae-49478f32d570 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:36.392428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:36.413733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d17dc-6937-4332-866e-c72a7c83c64e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.448524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770e345-a9d2-46e5-ac0d-450e90e3a289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.449761 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770e345-a9d2-46e5-ac0d-450e90e3a289 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.450035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770e345-a9d2-46e5-ac0d-450e90e3a289 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.450296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770e345-a9d2-46e5-ac0d-450e90e3a289 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.461367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770e345-a9d2-46e5-ac0d-450e90e3a289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4770e345-a9d2-46e5-ac0d-450e90e3a289 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:36.466521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:36.467433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:36.468276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:36.470007 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 395/796] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:36.471172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770e345-a9d2-46e5-ac0d-450e90e3a289 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:36.491085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:36.517972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceb4a4-144c-4933-bbae-49478f32d570 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.562009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38b3e1-637c-4319-af5e-e4826bba3508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.562220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38b3e1-637c-4319-af5e-e4826bba3508 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.562677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38b3e1-637c-4319-af5e-e4826bba3508 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.562803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38b3e1-637c-4319-af5e-e4826bba3508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.576101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38b3e1-637c-4319-af5e-e4826bba3508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:36.578188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38b3e1-637c-4319-af5e-e4826bba3508 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1e38b3e1-637c-4319-af5e-e4826bba3508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:36.597308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:36.598260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:36.599154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:36.599154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:36.599154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:36.603261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 402/797] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:36.608161 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38b3e1-637c-4319-af5e-e4826bba3508 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:36.608600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770e345-a9d2-46e5-ac0d-450e90e3a289 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.651544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9090b3a-9bce-4845-a90f-4c60d8ab5e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.652726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9090b3a-9bce-4845-a90f-4c60d8ab5e12 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.653979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9090b3a-9bce-4845-a90f-4c60d8ab5e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.654529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9090b3a-9bce-4845-a90f-4c60d8ab5e12 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.655387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9090b3a-9bce-4845-a90f-4c60d8ab5e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:36.655884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9090b3a-9bce-4845-a90f-4c60d8ab5e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:36.657454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9090b3a-9bce-4845-a90f-4c60d8ab5e12 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:36.672896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 396/798] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:36.674002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9090b3a-9bce-4845-a90f-4c60d8ab5e12 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:36.682380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:36.741226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38b3e1-637c-4319-af5e-e4826bba3508 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.764188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:36.783443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5ba1f-6ea5-4aeb-b507-6722aaa4c3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.784180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5ba1f-6ea5-4aeb-b507-6722aaa4c3ee None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.784402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5ba1f-6ea5-4aeb-b507-6722aaa4c3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.784614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5ba1f-6ea5-4aeb-b507-6722aaa4c3ee None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.785349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5ba1f-6ea5-4aeb-b507-6722aaa4c3ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:36.785531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5ba1f-6ea5-4aeb-b507-6722aaa4c3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:36.786855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5ba1f-6ea5-4aeb-b507-6722aaa4c3ee None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:36.797026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 403/799] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:36.798264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5ba1f-6ea5-4aeb-b507-6722aaa4c3ee None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:36.804064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9090b3a-9bce-4845-a90f-4c60d8ab5e12 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.840402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261de08-f2e7-4b92-aee8-7a90ec7bed62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.841737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261de08-f2e7-4b92-aee8-7a90ec7bed62 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.842006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261de08-f2e7-4b92-aee8-7a90ec7bed62 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.842264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261de08-f2e7-4b92-aee8-7a90ec7bed62 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.842890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261de08-f2e7-4b92-aee8-7a90ec7bed62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:36.843178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261de08-f2e7-4b92-aee8-7a90ec7bed62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:36.844534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261de08-f2e7-4b92-aee8-7a90ec7bed62 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:36.861409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 397/800] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:36.861409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a824092-54fa-47a7-8f42-4240e39b30d5 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.861409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a824092-54fa-47a7-8f42-4240e39b30d5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.861409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a824092-54fa-47a7-8f42-4240e39b30d5 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.861409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 398/801] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:36.861409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a824092-54fa-47a7-8f42-4240e39b30d5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:36.918540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5ba1f-6ea5-4aeb-b507-6722aaa4c3ee None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.934865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:36.952573 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03743287-06e4-4655-9822-8425e64dcc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:36.954420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03743287-06e4-4655-9822-8425e64dcc3f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:36.954420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03743287-06e4-4655-9822-8425e64dcc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:36.954420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03743287-06e4-4655-9822-8425e64dcc3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:36.961580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03743287-06e4-4655-9822-8425e64dcc3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:36.963025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03743287-06e4-4655-9822-8425e64dcc3f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:36.967317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a824092-54fa-47a7-8f42-4240e39b30d5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-03743287-06e4-4655-9822-8425e64dcc3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:36.975969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:36.977207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:36.978382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:36.978382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:36.978382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:36.978382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:36.978382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:36.978382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:36.978382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:36.978382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:36.981356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 404/802] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:36.982028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03743287-06e4-4655-9822-8425e64dcc3f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.005039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:37.010955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3506a28-529a-4f1f-bef2-ceb417c305ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.013126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3506a28-529a-4f1f-bef2-ceb417c305ea None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.013126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3506a28-529a-4f1f-bef2-ceb417c305ea None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.013126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3506a28-529a-4f1f-bef2-ceb417c305ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.021988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3506a28-529a-4f1f-bef2-ceb417c305ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.023438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3506a28-529a-4f1f-bef2-ceb417c305ea None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d3506a28-529a-4f1f-bef2-ceb417c305ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:37.038870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:37.039866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:37.040830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:37.040830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:37.040830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:37.040830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 399/803] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:37.041055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca70f90-0a8e-4ecc-9a3b-2cfb346140c2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.041751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca70f90-0a8e-4ecc-9a3b-2cfb346140c2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.041751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca70f90-0a8e-4ecc-9a3b-2cfb346140c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.059425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2ca70f90-0a8e-4ecc-9a3b-2cfb346140c2 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:37.060600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2ca70f90-0a8e-4ecc-9a3b-2cfb346140c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:37.094044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03743287-06e4-4655-9822-8425e64dcc3f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.136531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 400/804] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:37.136531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca70f90-0a8e-4ecc-9a3b-2cfb346140c2 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.139548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964c4a8-78a6-409a-8183-bcca3b58c9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.140228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964c4a8-78a6-409a-8183-bcca3b58c9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.140516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964c4a8-78a6-409a-8183-bcca3b58c9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.140843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964c4a8-78a6-409a-8183-bcca3b58c9a4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.149746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964c4a8-78a6-409a-8183-bcca3b58c9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.151236 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964c4a8-78a6-409a-8183-bcca3b58c9a4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f964c4a8-78a6-409a-8183-bcca3b58c9a4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:37.163815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:37.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 405/805] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:37.166262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964c4a8-78a6-409a-8183-bcca3b58c9a4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.210000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:37.241913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca70f90-0a8e-4ecc-9a3b-2cfb346140c2 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.258722 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:37.279488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ca339-994e-4daf-b8a1-f860e2fd1f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.280185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ca339-994e-4daf-b8a1-f860e2fd1f40 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.280461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ca339-994e-4daf-b8a1-f860e2fd1f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.280757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ca339-994e-4daf-b8a1-f860e2fd1f40 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.281338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964c4a8-78a6-409a-8183-bcca3b58c9a4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.281446 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ca339-994e-4daf-b8a1-f860e2fd1f40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:37.281726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ca339-994e-4daf-b8a1-f860e2fd1f40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.283136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ca339-994e-4daf-b8a1-f860e2fd1f40 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.294791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 401/806] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:37.295593 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ca339-994e-4daf-b8a1-f860e2fd1f40 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.315204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63d0b5-71b4-49ac-a0e3-774d5ed75893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.315871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63d0b5-71b4-49ac-a0e3-774d5ed75893 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.316211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63d0b5-71b4-49ac-a0e3-774d5ed75893 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.316565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63d0b5-71b4-49ac-a0e3-774d5ed75893 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.317197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63d0b5-71b4-49ac-a0e3-774d5ed75893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:37.317490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63d0b5-71b4-49ac-a0e3-774d5ed75893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.318868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63d0b5-71b4-49ac-a0e3-774d5ed75893 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.330897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 406/807] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:37.331669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63d0b5-71b4-49ac-a0e3-774d5ed75893 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.394314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ca339-994e-4daf-b8a1-f860e2fd1f40 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.423474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:37.427541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adec0f-e98e-4011-afde-eb5d05495bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.428161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adec0f-e98e-4011-afde-eb5d05495bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.428438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adec0f-e98e-4011-afde-eb5d05495bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.428741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adec0f-e98e-4011-afde-eb5d05495bc5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.428842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63d0b5-71b4-49ac-a0e3-774d5ed75893 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.429312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adec0f-e98e-4011-afde-eb5d05495bc5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.430673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adec0f-e98e-4011-afde-eb5d05495bc5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-96adec0f-e98e-4011-afde-eb5d05495bc5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:37.439124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:37.440144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:37.441021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 402/808] 10.222.0.50 () {56 vars in 1129 bytes} [Mon Aug 8 17:51:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:37.441821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057e5ff-b262-499d-b6a7-32721521e7ce None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.441821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057e5ff-b262-499d-b6a7-32721521e7ce None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.441821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057e5ff-b262-499d-b6a7-32721521e7ce None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.443034 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 403/809] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:37.443698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057e5ff-b262-499d-b6a7-32721521e7ce None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.467122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13965f20-b21f-407c-b9a5-fbc2fb531158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.467719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13965f20-b21f-407c-b9a5-fbc2fb531158 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.467990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13965f20-b21f-407c-b9a5-fbc2fb531158 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.468256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13965f20-b21f-407c-b9a5-fbc2fb531158 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.469002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13965f20-b21f-407c-b9a5-fbc2fb531158 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:37.469302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13965f20-b21f-407c-b9a5-fbc2fb531158 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.470738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13965f20-b21f-407c-b9a5-fbc2fb531158 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.484905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:37.488375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 407/810] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 17:51:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:37.489194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13965f20-b21f-407c-b9a5-fbc2fb531158 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.548367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057e5ff-b262-499d-b6a7-32721521e7ce None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.581634 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a783f0b-c580-4959-99c0-2c4097016f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.582279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a783f0b-c580-4959-99c0-2c4097016f1e None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.582548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a783f0b-c580-4959-99c0-2c4097016f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.582811 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a783f0b-c580-4959-99c0-2c4097016f1e None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/4f0ca85ea7974962b8115428563aef8f/roles/2978a8a56cd84b4b852dc6738c5c54c3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.588202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13965f20-b21f-407c-b9a5-fbc2fb531158 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.597505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a783f0b-c580-4959-99c0-2c4097016f1e None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=4f0ca85ea7974962b8115428563aef8f, role_id=2978a8a56cd84b4b852dc6738c5c54c3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.598927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a783f0b-c580-4959-99c0-2c4097016f1e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.623282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86391e64-86de-40dd-8cc1-838f69083e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.623906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86391e64-86de-40dd-8cc1-838f69083e1f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.624181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86391e64-86de-40dd-8cc1-838f69083e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.624446 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86391e64-86de-40dd-8cc1-838f69083e1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.631379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86391e64-86de-40dd-8cc1-838f69083e1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.632944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86391e64-86de-40dd-8cc1-838f69083e1f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-86391e64-86de-40dd-8cc1-838f69083e1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:37.647754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:37.648892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:37.650087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 408/811] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 17:51:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:37.650812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86391e64-86de-40dd-8cc1-838f69083e1f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.661911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 404/812] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:51:37 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/4f0ca85ea7974962b8115428563aef8f/roles/2978a8a56cd84b4b852dc6738c5c54c3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:51:37.663196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93445011-0405-46ac-ae48-c3fdae895339 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.663477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93445011-0405-46ac-ae48-c3fdae895339 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.663744 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93445011-0405-46ac-ae48-c3fdae895339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.684192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-93445011-0405-46ac-ae48-c3fdae895339 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:37.685645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-93445011-0405-46ac-ae48-c3fdae895339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:37.685759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:37.761065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 405/813] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:37.762955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93445011-0405-46ac-ae48-c3fdae895339 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.777259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86391e64-86de-40dd-8cc1-838f69083e1f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.795029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:37.825466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f5923-0fe5-414a-bee2-f9f9c68ce0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.826171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f5923-0fe5-414a-bee2-f9f9c68ce0bb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.826520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f5923-0fe5-414a-bee2-f9f9c68ce0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.826869 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f5923-0fe5-414a-bee2-f9f9c68ce0bb None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.827552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f5923-0fe5-414a-bee2-f9f9c68ce0bb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:37.827928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f5923-0fe5-414a-bee2-f9f9c68ce0bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.829589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f5923-0fe5-414a-bee2-f9f9c68ce0bb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.834591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 409/814] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 17:51:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:37.839946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f5923-0fe5-414a-bee2-f9f9c68ce0bb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.893289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93445011-0405-46ac-ae48-c3fdae895339 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.925493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:37.928608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a6bce-72ea-4122-9779-baf35b983007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.929431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a6bce-72ea-4122-9779-baf35b983007 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.929558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a6bce-72ea-4122-9779-baf35b983007 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.929756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a6bce-72ea-4122-9779-baf35b983007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.940904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a6bce-72ea-4122-9779-baf35b983007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.942338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a6bce-72ea-4122-9779-baf35b983007 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:37.946639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f5923-0fe5-414a-bee2-f9f9c68ce0bb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d23a6bce-72ea-4122-9779-baf35b983007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:37.957923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:37.958961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:37.959853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:37.959853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:37.959853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:37.959853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 406/815] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:37.959853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a6bce-72ea-4122-9779-baf35b983007 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:37.988602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5271e4-2b13-4744-a47a-690998afd388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:37.989205 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5271e4-2b13-4744-a47a-690998afd388 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:37.989475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5271e4-2b13-4744-a47a-690998afd388 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:37.989736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5271e4-2b13-4744-a47a-690998afd388 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:37.990451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5271e4-2b13-4744-a47a-690998afd388 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:37.990742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5271e4-2b13-4744-a47a-690998afd388 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:37.992159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5271e4-2b13-4744-a47a-690998afd388 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.010441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 410/816] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 17:51:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:38.011256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5271e4-2b13-4744-a47a-690998afd388 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.036048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:38.068872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a6bce-72ea-4122-9779-baf35b983007 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:38.106165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d57a5-9d23-4f9f-beba-1c11d712e20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:38.106795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d57a5-9d23-4f9f-beba-1c11d712e20f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:38.107070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d57a5-9d23-4f9f-beba-1c11d712e20f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:38.107343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d57a5-9d23-4f9f-beba-1c11d712e20f None admin] PATH_INFO: `/v3/services/network` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:38.107917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d57a5-9d23-4f9f-beba-1c11d712e20f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:38.109305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d57a5-9d23-4f9f-beba-1c11d712e20f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4d0d57a5-9d23-4f9f-beba-1c11d712e20f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:38.117973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:38.119174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:38.120445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 407/817] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:37 2022] GET /identity/v3/services/network => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:38.122302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d57a5-9d23-4f9f-beba-1c11d712e20f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.128266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5271e4-2b13-4744-a47a-690998afd388 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:38.162855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106706c-fe25-4997-824a-0db22ffbbdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:38.163679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106706c-fe25-4997-824a-0db22ffbbdb2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:38.163953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106706c-fe25-4997-824a-0db22ffbbdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:38.164212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106706c-fe25-4997-824a-0db22ffbbdb2 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:38.164984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106706c-fe25-4997-824a-0db22ffbbdb2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:38.165283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106706c-fe25-4997-824a-0db22ffbbdb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:38.166627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106706c-fe25-4997-824a-0db22ffbbdb2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.169228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:38.177385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 411/818] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 17:51:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:38.178189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106706c-fe25-4997-824a-0db22ffbbdb2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.227924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d57a5-9d23-4f9f-beba-1c11d712e20f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:38.265001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bacedc-385a-4320-a38f-8b4cc281107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:38.265862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bacedc-385a-4320-a38f-8b4cc281107e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:38.265976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bacedc-385a-4320-a38f-8b4cc281107e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:38.266243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bacedc-385a-4320-a38f-8b4cc281107e None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:38.266883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bacedc-385a-4320-a38f-8b4cc281107e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:38.267182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bacedc-385a-4320-a38f-8b4cc281107e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:38.268611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bacedc-385a-4320-a38f-8b4cc281107e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.270221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:38.285665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 408/819] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:38.285665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bacedc-385a-4320-a38f-8b4cc281107e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.290173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106706c-fe25-4997-824a-0db22ffbbdb2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:38.325530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cac4a-72a0-4759-af7f-98c5c0845d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:38.326045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cac4a-72a0-4759-af7f-98c5c0845d5b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:38.326320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cac4a-72a0-4759-af7f-98c5c0845d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:38.326585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cac4a-72a0-4759-af7f-98c5c0845d5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:38.393377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:38.395464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bacedc-385a-4320-a38f-8b4cc281107e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:38.395898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cac4a-72a0-4759-af7f-98c5c0845d5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:38.397455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cac4a-72a0-4759-af7f-98c5c0845d5b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.435814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f84c14-9286-452f-98c3-a374d7aa65ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:38.436414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f84c14-9286-452f-98c3-a374d7aa65ab None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:38.436716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f84c14-9286-452f-98c3-a374d7aa65ab None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:38.436984 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f84c14-9286-452f-98c3-a374d7aa65ab None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:38.437779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f84c14-9286-452f-98c3-a374d7aa65ab None admin] RBAC: Adding query filter params (name=network) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:38.437917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f84c14-9286-452f-98c3-a374d7aa65ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:38.439231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f84c14-9286-452f-98c3-a374d7aa65ab None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.448198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 409/820] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:38.448973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f84c14-9286-452f-98c3-a374d7aa65ab None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.502117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 412/821] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:51:38.502890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cac4a-72a0-4759-af7f-98c5c0845d5b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.515936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:38.560778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f84c14-9286-452f-98c3-a374d7aa65ab None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:38.599575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930edaf6-e70e-4f7f-b4fe-0b6d255b77cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:38.600187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930edaf6-e70e-4f7f-b4fe-0b6d255b77cf None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:38.600539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930edaf6-e70e-4f7f-b4fe-0b6d255b77cf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:38.600789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930edaf6-e70e-4f7f-b4fe-0b6d255b77cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:38.601398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930edaf6-e70e-4f7f-b4fe-0b6d255b77cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:38.602916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930edaf6-e70e-4f7f-b4fe-0b6d255b77cf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.603378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:38.607077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cac4a-72a0-4759-af7f-98c5c0845d5b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:38.642261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1e203-307b-4f27-93b1-d9968824f693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:38.642876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1e203-307b-4f27-93b1-d9968824f693 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:38.643148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1e203-307b-4f27-93b1-d9968824f693 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:38.643411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1e203-307b-4f27-93b1-d9968824f693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:38.649978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1e203-307b-4f27-93b1-d9968824f693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:38.651432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1e203-307b-4f27-93b1-d9968824f693 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-85d1e203-307b-4f27-93b1-d9968824f693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:38.662669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:38.665850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 413/822] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:38.666784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1e203-307b-4f27-93b1-d9968824f693 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.716532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 410/823] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 17:51:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:38.716532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930edaf6-e70e-4f7f-b4fe-0b6d255b77cf None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.789608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1e203-307b-4f27-93b1-d9968824f693 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:38.837872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812c3d1-8b0c-4d71-819b-c7033889bbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:38.838528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812c3d1-8b0c-4d71-819b-c7033889bbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:38.838839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812c3d1-8b0c-4d71-819b-c7033889bbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:38.839217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812c3d1-8b0c-4d71-819b-c7033889bbd7 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:38.840398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812c3d1-8b0c-4d71-819b-c7033889bbd7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:38.840718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812c3d1-8b0c-4d71-819b-c7033889bbd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:38.842093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812c3d1-8b0c-4d71-819b-c7033889bbd7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.852414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 414/824] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:38.852414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812c3d1-8b0c-4d71-819b-c7033889bbd7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.871764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930edaf6-e70e-4f7f-b4fe-0b6d255b77cf None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:38.878702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:38.894327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:38.919705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4456049-b0d5-477e-8b05-4e19355d24a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:38.920334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4456049-b0d5-477e-8b05-4e19355d24a0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:38.920652 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4456049-b0d5-477e-8b05-4e19355d24a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:38.920917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4456049-b0d5-477e-8b05-4e19355d24a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:38.936259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4456049-b0d5-477e-8b05-4e19355d24a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:38.937748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4456049-b0d5-477e-8b05-4e19355d24a0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d4456049-b0d5-477e-8b05-4e19355d24a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:38.954280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:38.955279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:38.956180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:38.956180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:38.956180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:38.956180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 411/825] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:38.956876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4456049-b0d5-477e-8b05-4e19355d24a0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:38.981495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812c3d1-8b0c-4d71-819b-c7033889bbd7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:39.017252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9ff80-570c-45b0-b846-55e9c47e1597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:39.017859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9ff80-570c-45b0-b846-55e9c47e1597 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:39.018170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9ff80-570c-45b0-b846-55e9c47e1597 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:39.018439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9ff80-570c-45b0-b846-55e9c47e1597 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:39.019127 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9ff80-570c-45b0-b846-55e9c47e1597 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:39.019431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9ff80-570c-45b0-b846-55e9c47e1597 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:39.020974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9ff80-570c-45b0-b846-55e9c47e1597 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:39.038461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 415/826] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:39.038461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9ff80-570c-45b0-b846-55e9c47e1597 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:39.082870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4456049-b0d5-477e-8b05-4e19355d24a0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:39.118642 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f4395-2281-44d3-8e55-8d236c58172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:39.119172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f4395-2281-44d3-8e55-8d236c58172d None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:39.119458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f4395-2281-44d3-8e55-8d236c58172d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:39.119724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f4395-2281-44d3-8e55-8d236c58172d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:39.120367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f4395-2281-44d3-8e55-8d236c58172d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:39.122423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f4395-2281-44d3-8e55-8d236c58172d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:39.126848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:39.147061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9ff80-570c-45b0-b846-55e9c47e1597 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:39.184527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 412/827] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 17:51:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:39.184527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f4395-2281-44d3-8e55-8d236c58172d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:39.217626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df404db-8b85-4544-8c25-4b1f86302e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:39.218242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df404db-8b85-4544-8c25-4b1f86302e37 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:39.218518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df404db-8b85-4544-8c25-4b1f86302e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:39.220336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df404db-8b85-4544-8c25-4b1f86302e37 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:39.221027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df404db-8b85-4544-8c25-4b1f86302e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:39.222208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df404db-8b85-4544-8c25-4b1f86302e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:39.225993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df404db-8b85-4544-8c25-4b1f86302e37 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:39.235513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:39.251558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 416/828] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:39.257237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df404db-8b85-4544-8c25-4b1f86302e37 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:39.393514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f4395-2281-44d3-8e55-8d236c58172d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:39.423695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df404db-8b85-4544-8c25-4b1f86302e37 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:39.447437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893091b-4752-4d97-b7e7-b31e0809ec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:39.448472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893091b-4752-4d97-b7e7-b31e0809ec8b None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:39.449995 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893091b-4752-4d97-b7e7-b31e0809ec8b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:39.450440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893091b-4752-4d97-b7e7-b31e0809ec8b None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/3375c3134ebe473cb5da29f083321cdb/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:39.453405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893091b-4752-4d97-b7e7-b31e0809ec8b None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=3375c3134ebe473cb5da29f083321cdb, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:39.455224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893091b-4752-4d97-b7e7-b31e0809ec8b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:39.478098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0942f-8202-4b50-a10f-debe7e06020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:39.478098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0942f-8202-4b50-a10f-debe7e06020b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:39.478098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0942f-8202-4b50-a10f-debe7e06020b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:39.478098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0942f-8202-4b50-a10f-debe7e06020b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:39.482859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:39.489552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0942f-8202-4b50-a10f-debe7e06020b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:39.491121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0942f-8202-4b50-a10f-debe7e06020b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b4b0942f-8202-4b50-a10f-debe7e06020b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:39.503247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:39.512460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 417/829] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:39.513481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0942f-8202-4b50-a10f-debe7e06020b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:39.535255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 413/830] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:51:39 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/3375c3134ebe473cb5da29f083321cdb/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:51:39.639028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0942f-8202-4b50-a10f-debe7e06020b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:39.678354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f8212-a289-4455-a0d3-fcccad39000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:39.678888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f8212-a289-4455-a0d3-fcccad39000e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:39.679164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f8212-a289-4455-a0d3-fcccad39000e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:39.679434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f8212-a289-4455-a0d3-fcccad39000e None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:39.680141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f8212-a289-4455-a0d3-fcccad39000e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:39.680440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f8212-a289-4455-a0d3-fcccad39000e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:39.681945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f8212-a289-4455-a0d3-fcccad39000e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:39.696725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 418/831] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:39.702878 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893091b-4752-4d97-b7e7-b31e0809ec8b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:39.741914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd516c89-cdfe-4c98-b876-c34bd45de12b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:39.742235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd516c89-cdfe-4c98-b876-c34bd45de12b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:39.742438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd516c89-cdfe-4c98-b876-c34bd45de12b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:39.743827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 419/832] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:39.749541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd674fa-c2a0-43d3-8e09-e2cc6cf21fb0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:39.749814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd674fa-c2a0-43d3-8e09-e2cc6cf21fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:39.750077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd674fa-c2a0-43d3-8e09-e2cc6cf21fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:39.768401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4bd674fa-c2a0-43d3-8e09-e2cc6cf21fb0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:39.769858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4bd674fa-c2a0-43d3-8e09-e2cc6cf21fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:39.794849 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:39.814355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893091b-4752-4d97-b7e7-b31e0809ec8b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:39.848661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:39.849900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86364c-fc9a-4340-b634-e50043211e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:39.850580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86364c-fc9a-4340-b634-e50043211e2f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:39.850919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86364c-fc9a-4340-b634-e50043211e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:39.851195 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86364c-fc9a-4340-b634-e50043211e2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:39.851977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86364c-fc9a-4340-b634-e50043211e2f None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=c496da229db34f90a4b3fdcfa582dca8) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:39.852291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86364c-fc9a-4340-b634-e50043211e2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:39.854053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86364c-fc9a-4340-b634-e50043211e2f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:39.861150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 414/833] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:39 2022] GET /identity/v3/role_assignments?user.id=c496da229db34f90a4b3fdcfa582dca8&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:39.872886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 420/834] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:51:39.896407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86364c-fc9a-4340-b634-e50043211e2f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:40.013688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86364c-fc9a-4340-b634-e50043211e2f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:40.048981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70acbd3-9057-4aa6-9021-44d63912ab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:40.049981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70acbd3-9057-4aa6-9021-44d63912ab15 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:40.050383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70acbd3-9057-4aa6-9021-44d63912ab15 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:40.050790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70acbd3-9057-4aa6-9021-44d63912ab15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:40.061534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70acbd3-9057-4aa6-9021-44d63912ab15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a70acbd3-9057-4aa6-9021-44d63912ab15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:40.066777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:40.067892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:51:40.068968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:40.071199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 415/835] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:51:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:40.077369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd674fa-c2a0-43d3-8e09-e2cc6cf21fb0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:40.182728 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd674fa-c2a0-43d3-8e09-e2cc6cf21fb0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:40.217949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d577c1c-ca4e-4eef-909b-042e82a699ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:40.218550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d577c1c-ca4e-4eef-909b-042e82a699ce None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:40.218824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d577c1c-ca4e-4eef-909b-042e82a699ce None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:40.219090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d577c1c-ca4e-4eef-909b-042e82a699ce None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:40.219743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d577c1c-ca4e-4eef-909b-042e82a699ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:40.220038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d577c1c-ca4e-4eef-909b-042e82a699ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:40.221431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d577c1c-ca4e-4eef-909b-042e82a699ce None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:40.230058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 421/836] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:51:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:40.235393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70acbd3-9057-4aa6-9021-44d63912ab15 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:40.252857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:40.338100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70acbd3-9057-4aa6-9021-44d63912ab15 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:40.373063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9543fa-2358-4a05-bdc8-b0b780d3345e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:40.374145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9543fa-2358-4a05-bdc8-b0b780d3345e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:40.374414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9543fa-2358-4a05-bdc8-b0b780d3345e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:40.374676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9543fa-2358-4a05-bdc8-b0b780d3345e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:40.385426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9543fa-2358-4a05-bdc8-b0b780d3345e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:40.388393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9543fa-2358-4a05-bdc8-b0b780d3345e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4a9543fa-2358-4a05-bdc8-b0b780d3345e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:40.403339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:40.404342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:40.405275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:40.405275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:40.405275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:40.408878 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 416/837] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:40.410413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d577c1c-ca4e-4eef-909b-042e82a699ce None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:40.462047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:40.515142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d577c1c-ca4e-4eef-909b-042e82a699ce None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:40.549465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62d9b7-b175-4ca4-974b-05d03b208a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:40.550125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62d9b7-b175-4ca4-974b-05d03b208a57 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:40.550406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62d9b7-b175-4ca4-974b-05d03b208a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:40.550674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62d9b7-b175-4ca4-974b-05d03b208a57 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:40.551309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62d9b7-b175-4ca4-974b-05d03b208a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:40.551606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62d9b7-b175-4ca4-974b-05d03b208a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:40.552996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62d9b7-b175-4ca4-974b-05d03b208a57 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:40.565857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 422/838] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:40.571388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9543fa-2358-4a05-bdc8-b0b780d3345e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:40.637371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:40.679815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9543fa-2358-4a05-bdc8-b0b780d3345e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:40.714117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111709f1-dfdd-4a63-be20-5d500e12387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:40.715248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111709f1-dfdd-4a63-be20-5d500e12387d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:40.715517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111709f1-dfdd-4a63-be20-5d500e12387d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:40.715832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111709f1-dfdd-4a63-be20-5d500e12387d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:40.723614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111709f1-dfdd-4a63-be20-5d500e12387d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:40.725179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111709f1-dfdd-4a63-be20-5d500e12387d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-111709f1-dfdd-4a63-be20-5d500e12387d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:40.736914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:40.739153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:40.741510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 417/839] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:51:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:40.745260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62d9b7-b175-4ca4-974b-05d03b208a57 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:40.859942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62d9b7-b175-4ca4-974b-05d03b208a57 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:40.869486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:40.894395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb3910-1258-4df0-a397-0159de9a5b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:40.894914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb3910-1258-4df0-a397-0159de9a5b6d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:40.895188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb3910-1258-4df0-a397-0159de9a5b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:40.895449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb3910-1258-4df0-a397-0159de9a5b6d None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:40.896153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb3910-1258-4df0-a397-0159de9a5b6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:40.896444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb3910-1258-4df0-a397-0159de9a5b6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:40.897968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb3910-1258-4df0-a397-0159de9a5b6d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:40.916067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4f486-dfa5-4391-8da8-e2249b6cf8a2 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:40.916586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4f486-dfa5-4391-8da8-e2249b6cf8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:40.916586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4f486-dfa5-4391-8da8-e2249b6cf8a2 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:40.918669 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 418/840] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:40.920698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 423/841] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 17:51:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:40.926094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c254a-4f28-4a84-a497-bd07c1247239 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:40.926591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c254a-4f28-4a84-a497-bd07c1247239 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:40.926784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb3910-1258-4df0-a397-0159de9a5b6d None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:40.927239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c254a-4f28-4a84-a497-bd07c1247239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:40.945838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-813c254a-4f28-4a84-a497-bd07c1247239 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:40.947501 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-813c254a-4f28-4a84-a497-bd07c1247239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:41.022560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 419/842] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:51:41.042708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb3910-1258-4df0-a397-0159de9a5b6d None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:41.044689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c254a-4f28-4a84-a497-bd07c1247239 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:41.085214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff5fec-807a-4e1e-b2ff-6629ac0f6787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:41.086606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff5fec-807a-4e1e-b2ff-6629ac0f6787 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:41.086883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff5fec-807a-4e1e-b2ff-6629ac0f6787 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:41.087155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff5fec-807a-4e1e-b2ff-6629ac0f6787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:41.101215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:41.132293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:41.167292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c254a-4f28-4a84-a497-bd07c1247239 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:41.172726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff5fec-807a-4e1e-b2ff-6629ac0f6787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:41.174384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff5fec-807a-4e1e-b2ff-6629ac0f6787 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:41.212856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693506b0-5efd-4801-a18b-da992a4507be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:41.213491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693506b0-5efd-4801-a18b-da992a4507be None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:41.213769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693506b0-5efd-4801-a18b-da992a4507be None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:41.214038 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693506b0-5efd-4801-a18b-da992a4507be None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:41.215188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693506b0-5efd-4801-a18b-da992a4507be None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:41.216576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693506b0-5efd-4801-a18b-da992a4507be None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-693506b0-5efd-4801-a18b-da992a4507be None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:41.228098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:41.229240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 17:51:41.230098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:41.232870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 420/843] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:41.239989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693506b0-5efd-4801-a18b-da992a4507be None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:41.296282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 424/844] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:51:41.301719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff5fec-807a-4e1e-b2ff-6629ac0f6787 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:41.362352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693506b0-5efd-4801-a18b-da992a4507be None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:41.375759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:41.392863 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:41.397745 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d876d67-68ef-4821-8599-b11e24f4a5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:41.398125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d876d67-68ef-4821-8599-b11e24f4a5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:41.398396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d876d67-68ef-4821-8599-b11e24f4a5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:41.398662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d876d67-68ef-4821-8599-b11e24f4a5e8 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:41.399291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d876d67-68ef-4821-8599-b11e24f4a5e8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:41.399582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d876d67-68ef-4821-8599-b11e24f4a5e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:41.401108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d876d67-68ef-4821-8599-b11e24f4a5e8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:41.410538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 421/845] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:41.411385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d876d67-68ef-4821-8599-b11e24f4a5e8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:41.414057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff5fec-807a-4e1e-b2ff-6629ac0f6787 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:41.452313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4736ef9-0dfb-4cf9-a2a4-a2e1182ec4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:41.453352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4736ef9-0dfb-4cf9-a2a4-a2e1182ec4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:41.454373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4736ef9-0dfb-4cf9-a2a4-a2e1182ec4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:41.454785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4736ef9-0dfb-4cf9-a2a4-a2e1182ec4f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:41.465969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4736ef9-0dfb-4cf9-a2a4-a2e1182ec4f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:41.467783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4736ef9-0dfb-4cf9-a2a4-a2e1182ec4f0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f4736ef9-0dfb-4cf9-a2a4-a2e1182ec4f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:41.483262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:41.484259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:41.485186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:41.485186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:41.485186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:41.491553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 425/846] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:41.492792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4736ef9-0dfb-4cf9-a2a4-a2e1182ec4f0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:41.516707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d876d67-68ef-4821-8599-b11e24f4a5e8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:41.551753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c225-81ea-454d-9174-04cb114c1c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:41.552434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c225-81ea-454d-9174-04cb114c1c48 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:41.552721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c225-81ea-454d-9174-04cb114c1c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:41.552979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c225-81ea-454d-9174-04cb114c1c48 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:41.553604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636c225-81ea-454d-9174-04cb114c1c48 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:41.553899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636c225-81ea-454d-9174-04cb114c1c48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:41.555447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636c225-81ea-454d-9174-04cb114c1c48 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:41.564964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 422/847] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:41.604016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4736ef9-0dfb-4cf9-a2a4-a2e1182ec4f0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:41.624409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:41.643652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e572a4-ae2e-400d-85dc-d89533b910df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:41.644541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e572a4-ae2e-400d-85dc-d89533b910df None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:41.644988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e572a4-ae2e-400d-85dc-d89533b910df None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:41.645397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e572a4-ae2e-400d-85dc-d89533b910df None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:41.646264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e572a4-ae2e-400d-85dc-d89533b910df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:41.646706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e572a4-ae2e-400d-85dc-d89533b910df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:41.648248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e572a4-ae2e-400d-85dc-d89533b910df None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:41.662284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 426/848] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:41.668162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c225-81ea-454d-9174-04cb114c1c48 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:41.720453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:41.722698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c49c75-a076-4878-bac7-404afc8e1c2f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:41.722986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c49c75-a076-4878-bac7-404afc8e1c2f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:41.723258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c49c75-a076-4878-bac7-404afc8e1c2f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:41.727788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 427/849] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:41.731993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7329c-e288-4436-9768-b87662b651c6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:41.732634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7329c-e288-4436-9768-b87662b651c6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:41.733032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7329c-e288-4436-9768-b87662b651c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:41.756857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fda7329c-e288-4436-9768-b87662b651c6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:41.758058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fda7329c-e288-4436-9768-b87662b651c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:41.790434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c225-81ea-454d-9174-04cb114c1c48 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:41.836819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed6668-1b51-4c24-be12-905617fcbfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:41.837861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed6668-1b51-4c24-be12-905617fcbfbf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:41.838357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed6668-1b51-4c24-be12-905617fcbfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:41.838820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed6668-1b51-4c24-be12-905617fcbfbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:41.838937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 428/850] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:51:41.846355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed6668-1b51-4c24-be12-905617fcbfbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:41.847892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed6668-1b51-4c24-be12-905617fcbfbf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-62ed6668-1b51-4c24-be12-905617fcbfbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:41.859726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:41.860657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:41.861592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:41.861592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:41.861592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:41.861592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:41.861592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:41.861592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:41.861592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:41.861592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:41.861592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 423/851] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:41.867912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7329c-e288-4436-9768-b87662b651c6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:41.870400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed6668-1b51-4c24-be12-905617fcbfbf None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:41.975385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7329c-e288-4436-9768-b87662b651c6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:41.985982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:42.010320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:42.013646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bc2c8-753a-40e2-87f5-07341b38490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.014387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bc2c8-753a-40e2-87f5-07341b38490c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.014738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed6668-1b51-4c24-be12-905617fcbfbf None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:42.014852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bc2c8-753a-40e2-87f5-07341b38490c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.015204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bc2c8-753a-40e2-87f5-07341b38490c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.028023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bc2c8-753a-40e2-87f5-07341b38490c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-af4bc2c8-753a-40e2-87f5-07341b38490c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:42.034820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:42.044037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 429/852] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:42.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bc2c8-753a-40e2-87f5-07341b38490c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:42.067168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb982e-6b63-490c-9a9b-b88974497e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.067884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb982e-6b63-490c-9a9b-b88974497e80 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.068162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb982e-6b63-490c-9a9b-b88974497e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.068431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb982e-6b63-490c-9a9b-b88974497e80 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.069170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb982e-6b63-490c-9a9b-b88974497e80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:42.069634 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb982e-6b63-490c-9a9b-b88974497e80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:42.071060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb982e-6b63-490c-9a9b-b88974497e80 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:42.083362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 424/853] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:42.084730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56c91-f7db-4a66-93cb-34f234f77e71 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.084994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56c91-f7db-4a66-93cb-34f234f77e71 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.085270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56c91-f7db-4a66-93cb-34f234f77e71 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.090832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 425/854] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:42.090832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56c91-f7db-4a66-93cb-34f234f77e71 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:42.159509 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bc2c8-753a-40e2-87f5-07341b38490c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:42.194324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5a4d5-efe1-45b5-b892-b4468ab170cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.194913 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5a4d5-efe1-45b5-b892-b4468ab170cc None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.195181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5a4d5-efe1-45b5-b892-b4468ab170cc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.195359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56c91-f7db-4a66-93cb-34f234f77e71 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:42.195461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5a4d5-efe1-45b5-b892-b4468ab170cc None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.196142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5a4d5-efe1-45b5-b892-b4468ab170cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:42.196431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5a4d5-efe1-45b5-b892-b4468ab170cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:42.197868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5a4d5-efe1-45b5-b892-b4468ab170cc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:42.206618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 430/855] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:42.207771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1f89-388d-4da9-892b-776ea71e277a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.208048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1f89-388d-4da9-892b-776ea71e277a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.208308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1f89-388d-4da9-892b-776ea71e277a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.227296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ed2d1f89-388d-4da9-892b-776ea71e277a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:42.228395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ed2d1f89-388d-4da9-892b-776ea71e277a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:42.238988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4885cb-8c4d-4e26-9643-ec11b47da453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.239954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4885cb-8c4d-4e26-9643-ec11b47da453 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.240290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4885cb-8c4d-4e26-9643-ec11b47da453 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.240596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4885cb-8c4d-4e26-9643-ec11b47da453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.241321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4885cb-8c4d-4e26-9643-ec11b47da453 None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=4f0ca85ea7974962b8115428563aef8f) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:42.241642 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4885cb-8c4d-4e26-9643-ec11b47da453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:42.243073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4885cb-8c4d-4e26-9643-ec11b47da453 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:42.249386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 426/856] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:42 2022] GET /identity/v3/role_assignments?user.id=4f0ca85ea7974962b8115428563aef8f&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:42.250189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4885cb-8c4d-4e26-9643-ec11b47da453 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:42.262436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:42.287047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:42.325049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 431/857] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:42.339191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1f89-388d-4da9-892b-776ea71e277a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:42.373368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4885cb-8c4d-4e26-9643-ec11b47da453 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:42.414073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d8b86-07a0-44f2-bdac-bc23f22eda91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.414974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d8b86-07a0-44f2-bdac-bc23f22eda91 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.415418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d8b86-07a0-44f2-bdac-bc23f22eda91 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.415977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d8b86-07a0-44f2-bdac-bc23f22eda91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.429278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d8b86-07a0-44f2-bdac-bc23f22eda91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:42.430961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d8b86-07a0-44f2-bdac-bc23f22eda91 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-202d8b86-07a0-44f2-bdac-bc23f22eda91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:42.446806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:42.447819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:42.449369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:42.449369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:42.449369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:42.449369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 427/858] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:42.454723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d8b86-07a0-44f2-bdac-bc23f22eda91 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:42.467439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1f89-388d-4da9-892b-776ea71e277a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:42.513706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410196c4-4bdb-4263-b420-fcf988ac8b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.514253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410196c4-4bdb-4263-b420-fcf988ac8b58 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.514522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410196c4-4bdb-4263-b420-fcf988ac8b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.514782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410196c4-4bdb-4263-b420-fcf988ac8b58 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.515357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410196c4-4bdb-4263-b420-fcf988ac8b58 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:42.516851 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410196c4-4bdb-4263-b420-fcf988ac8b58 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-410196c4-4bdb-4263-b420-fcf988ac8b58 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:42.528066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:42.529083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:42.530016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 432/859] 10.222.0.50 () {56 vars in 1132 bytes} [Mon Aug 8 17:51:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:42.535671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410196c4-4bdb-4263-b420-fcf988ac8b58 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:42.556495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:42.565296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:42.579012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d8b86-07a0-44f2-bdac-bc23f22eda91 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:42.620498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612a891-ed2f-435c-b927-5897b6852987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.621158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612a891-ed2f-435c-b927-5897b6852987 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.621448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612a891-ed2f-435c-b927-5897b6852987 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.621693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612a891-ed2f-435c-b927-5897b6852987 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.622325 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612a891-ed2f-435c-b927-5897b6852987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:42.622615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612a891-ed2f-435c-b927-5897b6852987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:42.624124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612a891-ed2f-435c-b927-5897b6852987 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:42.637475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 428/860] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:42.642898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612a891-ed2f-435c-b927-5897b6852987 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:42.667684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410196c4-4bdb-4263-b420-fcf988ac8b58 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:42.712791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2fa62-4b65-4017-b6d1-bef1ae18ffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.713395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2fa62-4b65-4017-b6d1-bef1ae18ffb4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.713668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2fa62-4b65-4017-b6d1-bef1ae18ffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.713942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2fa62-4b65-4017-b6d1-bef1ae18ffb4 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.715125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2fa62-4b65-4017-b6d1-bef1ae18ffb4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:42.715425 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2fa62-4b65-4017-b6d1-bef1ae18ffb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:42.716843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2fa62-4b65-4017-b6d1-bef1ae18ffb4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:42.726387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 433/861] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:42.727422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2fa62-4b65-4017-b6d1-bef1ae18ffb4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:42.784496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612a891-ed2f-435c-b927-5897b6852987 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:42.821079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d4efb-752e-4e71-8a6a-d0c11aad3cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.821982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d4efb-752e-4e71-8a6a-d0c11aad3cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.822486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d4efb-752e-4e71-8a6a-d0c11aad3cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.822991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d4efb-752e-4e71-8a6a-d0c11aad3cc8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.831335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d4efb-752e-4e71-8a6a-d0c11aad3cc8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:42.833122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d4efb-752e-4e71-8a6a-d0c11aad3cc8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:42.835601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:42.836841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8f2d4efb-752e-4e71-8a6a-d0c11aad3cc8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:42.847840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:42.853046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 17:51:42.854036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:42.854681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2fa62-4b65-4017-b6d1-bef1ae18ffb4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:42.855517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 429/862] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:51:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:42.858611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d4efb-752e-4e71-8a6a-d0c11aad3cc8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:42.892490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b4d38-60ac-4c01-891c-6d361f9f684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:42.893168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b4d38-60ac-4c01-891c-6d361f9f684a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:42.893374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b4d38-60ac-4c01-891c-6d361f9f684a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:42.893640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b4d38-60ac-4c01-891c-6d361f9f684a None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:42.894259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b4d38-60ac-4c01-891c-6d361f9f684a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:42.894556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b4d38-60ac-4c01-891c-6d361f9f684a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:42.895908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b4d38-60ac-4c01-891c-6d361f9f684a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:42.905647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 434/863] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:42.974590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d4efb-752e-4e71-8a6a-d0c11aad3cc8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:43.017481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2300991-1354-40e1-876b-2d39672cd09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:43.018147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2300991-1354-40e1-876b-2d39672cd09c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:43.018362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2300991-1354-40e1-876b-2d39672cd09c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:43.018630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2300991-1354-40e1-876b-2d39672cd09c None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:43.019366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2300991-1354-40e1-876b-2d39672cd09c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:43.019663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2300991-1354-40e1-876b-2d39672cd09c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:43.021147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2300991-1354-40e1-876b-2d39672cd09c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:43.043461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 430/864] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 17:51:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:43.049334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b4d38-60ac-4c01-891c-6d361f9f684a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:43.159092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b4d38-60ac-4c01-891c-6d361f9f684a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:43.195184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41bddf-20b9-4940-93be-a95ddc2f734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:43.196172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41bddf-20b9-4940-93be-a95ddc2f734c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:43.196688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41bddf-20b9-4940-93be-a95ddc2f734c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:43.197155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41bddf-20b9-4940-93be-a95ddc2f734c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:43.220804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:43.270054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41bddf-20b9-4940-93be-a95ddc2f734c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:43.271506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41bddf-20b9-4940-93be-a95ddc2f734c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:43.375375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 435/865] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:51:43.381060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2300991-1354-40e1-876b-2d39672cd09c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:43.437869 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:43.482650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2300991-1354-40e1-876b-2d39672cd09c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:43.521075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ea791-5d30-492f-b08c-49177ea751e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:43.521977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea791-5d30-492f-b08c-49177ea751e7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:43.522389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea791-5d30-492f-b08c-49177ea751e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:43.522794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea791-5d30-492f-b08c-49177ea751e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:43.533488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ea791-5d30-492f-b08c-49177ea751e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:43.535137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ea791-5d30-492f-b08c-49177ea751e7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-333ea791-5d30-492f-b08c-49177ea751e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:43.550399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:43.551546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:43.552682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:43.552682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:43.552682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:43.554303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 431/866] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:43.560547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41bddf-20b9-4940-93be-a95ddc2f734c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:43.619067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:43.675622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785f37e-3a18-47a6-ac63-618279d4b29a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:43.675815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785f37e-3a18-47a6-ac63-618279d4b29a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:43.676066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785f37e-3a18-47a6-ac63-618279d4b29a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:43.677551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41bddf-20b9-4940-93be-a95ddc2f734c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:43.677794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 432/867] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:43.683208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c73c1-e2b6-4c0b-b01f-339358608bf7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:43.683493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c73c1-e2b6-4c0b-b01f-339358608bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:43.683761 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c73c1-e2b6-4c0b-b01f-339358608bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:43.702381 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4d5c73c1-e2b6-4c0b-b01f-339358608bf7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:43.703811 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4d5c73c1-e2b6-4c0b-b01f-339358608bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:43.717524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296eeed-2462-4cfb-ba57-3b57b9d38a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:43.718427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296eeed-2462-4cfb-ba57-3b57b9d38a57 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:43.718908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296eeed-2462-4cfb-ba57-3b57b9d38a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:43.719381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296eeed-2462-4cfb-ba57-3b57b9d38a57 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:43.720244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296eeed-2462-4cfb-ba57-3b57b9d38a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:43.720803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296eeed-2462-4cfb-ba57-3b57b9d38a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:43.722363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296eeed-2462-4cfb-ba57-3b57b9d38a57 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:43.735390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 436/868] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:43.743266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296eeed-2462-4cfb-ba57-3b57b9d38a57 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:43.780643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 433/869] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:43.807188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c73c1-e2b6-4c0b-b01f-339358608bf7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:43.844329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:43.845145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296eeed-2462-4cfb-ba57-3b57b9d38a57 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:43.858088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:43.879414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc9d2d-15c1-4088-acf4-17119ee178f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:43.880076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc9d2d-15c1-4088-acf4-17119ee178f9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:43.880352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc9d2d-15c1-4088-acf4-17119ee178f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:43.880647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc9d2d-15c1-4088-acf4-17119ee178f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:43.887992 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc9d2d-15c1-4088-acf4-17119ee178f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:43.889451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc9d2d-15c1-4088-acf4-17119ee178f9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-e7dc9d2d-15c1-4088-acf4-17119ee178f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:43.900943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:43.902287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:43.903193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:43.903193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:43.903193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:43.903193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:43.903193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:43.903193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:43.903193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:43.903193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:43.903193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 437/870] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:43.908860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc9d2d-15c1-4088-acf4-17119ee178f9 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:43.918695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c73c1-e2b6-4c0b-b01f-339358608bf7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:43.955129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662ddd6-135c-4d80-91db-2d1681f7c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:43.956143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662ddd6-135c-4d80-91db-2d1681f7c55d None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:43.956647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662ddd6-135c-4d80-91db-2d1681f7c55d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:43.957095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662ddd6-135c-4d80-91db-2d1681f7c55d None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:43.957925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662ddd6-135c-4d80-91db-2d1681f7c55d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:43.959469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662ddd6-135c-4d80-91db-2d1681f7c55d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:44.014509 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc9d2d-15c1-4088-acf4-17119ee178f9 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:44.029796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 434/871] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:51:43 2022] POST /identity/v3/services => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:44.069622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fda36-50fc-4e3a-8969-d8ba5e97e9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:44.070583 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fda36-50fc-4e3a-8969-d8ba5e97e9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:44.071086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fda36-50fc-4e3a-8969-d8ba5e97e9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:44.071539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fda36-50fc-4e3a-8969-d8ba5e97e9f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:44.072450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fda36-50fc-4e3a-8969-d8ba5e97e9f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:44.073003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fda36-50fc-4e3a-8969-d8ba5e97e9f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:44.074934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fda36-50fc-4e3a-8969-d8ba5e97e9f3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:44.091657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 438/872] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:44.098868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662ddd6-135c-4d80-91db-2d1681f7c55d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:44.209348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:44.218545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662ddd6-135c-4d80-91db-2d1681f7c55d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:44.255273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e4777-1904-4ab9-9602-78ff7507edaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:44.255870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e4777-1904-4ab9-9602-78ff7507edaf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:44.256140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e4777-1904-4ab9-9602-78ff7507edaf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:44.256401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e4777-1904-4ab9-9602-78ff7507edaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:44.257137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e4777-1904-4ab9-9602-78ff7507edaf None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=3375c3134ebe473cb5da29f083321cdb) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:44.257433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e4777-1904-4ab9-9602-78ff7507edaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:44.258837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e4777-1904-4ab9-9602-78ff7507edaf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:44.265621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 435/873] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:44 2022] GET /identity/v3/role_assignments?user.id=3375c3134ebe473cb5da29f083321cdb&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:44.542266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a668e0-6baf-4cc4-b742-d9941ebf4c2b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:44.542668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a668e0-6baf-4cc4-b742-d9941ebf4c2b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:44.542774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a668e0-6baf-4cc4-b742-d9941ebf4c2b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:44.544371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 439/874] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:44.549989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ae36a-a210-455f-9efa-7af456b4873e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:44.550308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ae36a-a210-455f-9efa-7af456b4873e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:44.550542 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ae36a-a210-455f-9efa-7af456b4873e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:44.569314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ef0ae36a-a210-455f-9efa-7af456b4873e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:44.570500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ef0ae36a-a210-455f-9efa-7af456b4873e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:44.645846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 436/875] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:44.668737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a668e0-6baf-4cc4-b742-d9941ebf4c2b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:44.699839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:44.777565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a668e0-6baf-4cc4-b742-d9941ebf4c2b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:44.813899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1295247-2083-4d3b-ada1-ab8098427a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:44.814527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1295247-2083-4d3b-ada1-ab8098427a08 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:44.814797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1295247-2083-4d3b-ada1-ab8098427a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:44.815073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1295247-2083-4d3b-ada1-ab8098427a08 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:44.826598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1295247-2083-4d3b-ada1-ab8098427a08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f1295247-2083-4d3b-ada1-ab8098427a08 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:44.830772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:44.831734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:44.832636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 440/876] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 17:51:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:44.838236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ae36a-a210-455f-9efa-7af456b4873e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:44.941353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ae36a-a210-455f-9efa-7af456b4873e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:44.954164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:44.982553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96197-e6e5-48bd-9dab-2429f9bab6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:44.983518 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96197-e6e5-48bd-9dab-2429f9bab6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:44.983999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96197-e6e5-48bd-9dab-2429f9bab6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:44.984484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96197-e6e5-48bd-9dab-2429f9bab6d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:44.985389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96197-e6e5-48bd-9dab-2429f9bab6d1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:44.985902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96197-e6e5-48bd-9dab-2429f9bab6d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:44.987533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96197-e6e5-48bd-9dab-2429f9bab6d1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:45.000616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 437/877] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 17:51:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:45.002989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1295247-2083-4d3b-ada1-ab8098427a08 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:45.028245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cbdfa-a0d9-4134-b76c-3804894e5d2f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:45.028565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cbdfa-a0d9-4134-b76c-3804894e5d2f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:45.028855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cbdfa-a0d9-4134-b76c-3804894e5d2f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:45.030307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 438/878] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:45.035752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f3ded-a62c-43b0-9bfc-11a18bebae0d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:45.036253 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f3ded-a62c-43b0-9bfc-11a18bebae0d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:45.036757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f3ded-a62c-43b0-9bfc-11a18bebae0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:45.056928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3e5f3ded-a62c-43b0-9bfc-11a18bebae0d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:45.058951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3e5f3ded-a62c-43b0-9bfc-11a18bebae0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:45.114441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:45.127015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1295247-2083-4d3b-ada1-ab8098427a08 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:45.156678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 439/879] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:51:45.164045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73c285-1529-413f-9682-01d3f77723ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:45.164710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73c285-1529-413f-9682-01d3f77723ea None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:45.164991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73c285-1529-413f-9682-01d3f77723ea None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:45.165265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73c285-1529-413f-9682-01d3f77723ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:45.174698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f3ded-a62c-43b0-9bfc-11a18bebae0d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:45.180454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73c285-1529-413f-9682-01d3f77723ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:45.182176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73c285-1529-413f-9682-01d3f77723ea None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5b73c285-1529-413f-9682-01d3f77723ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:45.199061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:45.200111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:45.201075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:45.201075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:45.201075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:45.203880 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 441/880] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:45.210182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73c285-1529-413f-9682-01d3f77723ea None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:45.280476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:45.302572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f3ded-a62c-43b0-9bfc-11a18bebae0d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:45.346540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590f123-68e7-4fd3-bdf9-8abbdfaf4466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:45.347070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590f123-68e7-4fd3-bdf9-8abbdfaf4466 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:45.347394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590f123-68e7-4fd3-bdf9-8abbdfaf4466 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:45.347602 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590f123-68e7-4fd3-bdf9-8abbdfaf4466 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:45.348257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590f123-68e7-4fd3-bdf9-8abbdfaf4466 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:45.349653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590f123-68e7-4fd3-bdf9-8abbdfaf4466 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:45.350356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73c285-1529-413f-9682-01d3f77723ea None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:45.386480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc1cae-836d-4f9c-8052-508a6b83ff8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:45.387379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc1cae-836d-4f9c-8052-508a6b83ff8c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:45.387875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc1cae-836d-4f9c-8052-508a6b83ff8c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:45.388343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc1cae-836d-4f9c-8052-508a6b83ff8c None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:45.389220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc1cae-836d-4f9c-8052-508a6b83ff8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:45.389733 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc1cae-836d-4f9c-8052-508a6b83ff8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:45.391297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc1cae-836d-4f9c-8052-508a6b83ff8c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:45.401038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:45.409597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 442/881] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:45.411003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 440/882] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:51:45 2022] POST /identity/v3/services => generated 269 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:51:45.413492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc1cae-836d-4f9c-8052-508a6b83ff8c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:45.529739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc1cae-836d-4f9c-8052-508a6b83ff8c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:45.573431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea4e0a-192d-4eb2-9639-0484e9d266fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:45.574841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea4e0a-192d-4eb2-9639-0484e9d266fe None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:45.575265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea4e0a-192d-4eb2-9639-0484e9d266fe None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:45.575705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea4e0a-192d-4eb2-9639-0484e9d266fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:45.583745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea4e0a-192d-4eb2-9639-0484e9d266fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:45.585263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea4e0a-192d-4eb2-9639-0484e9d266fe None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-ecea4e0a-192d-4eb2-9639-0484e9d266fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:45.597549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:45.598499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:45.599482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:45.600710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 443/883] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:51:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:45.604634 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590f123-68e7-4fd3-bdf9-8abbdfaf4466 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:45.713810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590f123-68e7-4fd3-bdf9-8abbdfaf4466 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:45.748604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e32957-bdd3-4647-b368-1a0c390c50d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:45.749213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32957-bdd3-4647-b368-1a0c390c50d5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:45.749488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32957-bdd3-4647-b368-1a0c390c50d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:45.749756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32957-bdd3-4647-b368-1a0c390c50d5 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:45.750482 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e32957-bdd3-4647-b368-1a0c390c50d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:45.750781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e32957-bdd3-4647-b368-1a0c390c50d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:45.752202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e32957-bdd3-4647-b368-1a0c390c50d5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:45.767191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:45.771838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 441/884] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 17:51:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:45.778263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea4e0a-192d-4eb2-9639-0484e9d266fe None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:45.852065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:45.900467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea4e0a-192d-4eb2-9639-0484e9d266fe None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:45.936182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781681f9-86dc-4232-8393-e0d9d9b19a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:45.936874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781681f9-86dc-4232-8393-e0d9d9b19a14 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:45.937150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781681f9-86dc-4232-8393-e0d9d9b19a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:45.937418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781681f9-86dc-4232-8393-e0d9d9b19a14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.010207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781681f9-86dc-4232-8393-e0d9d9b19a14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:46.012253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781681f9-86dc-4232-8393-e0d9d9b19a14 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:46.073469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:46.105061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5611f-da9f-41b2-af63-aabdf19b687a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.105547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5611f-da9f-41b2-af63-aabdf19b687a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.105547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5611f-da9f-41b2-af63-aabdf19b687a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.107066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 442/885] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:46.111891 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b90a1-aca4-4005-ae16-10ff9c6b2368 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.112165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b90a1-aca4-4005-ae16-10ff9c6b2368 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.112433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b90a1-aca4-4005-ae16-10ff9c6b2368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.120165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 444/886] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:45 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:51:46.126212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781681f9-86dc-4232-8393-e0d9d9b19a14 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:46.134359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-357b90a1-aca4-4005-ae16-10ff9c6b2368 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:46.135870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-357b90a1-aca4-4005-ae16-10ff9c6b2368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:46.228896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 443/887] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:51:46.241292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781681f9-86dc-4232-8393-e0d9d9b19a14 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:46.250499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b90a1-aca4-4005-ae16-10ff9c6b2368 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:46.291098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128efb-7e71-4e38-8760-d4cee0a7c6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:46.292071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128efb-7e71-4e38-8760-d4cee0a7c6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.293115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128efb-7e71-4e38-8760-d4cee0a7c6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.294082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128efb-7e71-4e38-8760-d4cee0a7c6c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.308706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05128efb-7e71-4e38-8760-d4cee0a7c6c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:46.311001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05128efb-7e71-4e38-8760-d4cee0a7c6c6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-05128efb-7e71-4e38-8760-d4cee0a7c6c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:46.332194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:46.333188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:46.334094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:46.334094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:46.334094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:46.341057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 445/888] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:46.342483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128efb-7e71-4e38-8760-d4cee0a7c6c6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:46.358980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:46.365751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b90a1-aca4-4005-ae16-10ff9c6b2368 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:46.398189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:46.404937 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f8d8a-39db-4d4f-938d-fe0ebcd472b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:46.405568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f8d8a-39db-4d4f-938d-fe0ebcd472b4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.405846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f8d8a-39db-4d4f-938d-fe0ebcd472b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.406115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f8d8a-39db-4d4f-938d-fe0ebcd472b4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.406693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f8d8a-39db-4d4f-938d-fe0ebcd472b4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:46.408153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f8d8a-39db-4d4f-938d-fe0ebcd472b4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-235f8d8a-39db-4d4f-938d-fe0ebcd472b4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:46.416716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:46.418246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:46.419441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 444/889] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:46.420246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5552b86-49a8-4e4d-9015-a58184fd96c0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.420246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5552b86-49a8-4e4d-9015-a58184fd96c0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.420246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5552b86-49a8-4e4d-9015-a58184fd96c0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.425078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 445/890] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:46.426191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5552b86-49a8-4e4d-9015-a58184fd96c0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:46.468412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128efb-7e71-4e38-8760-d4cee0a7c6c6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:46.502031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c4fbc-9282-4bba-8827-4555e78ba407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:46.502626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c4fbc-9282-4bba-8827-4555e78ba407 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.502901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c4fbc-9282-4bba-8827-4555e78ba407 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.503170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c4fbc-9282-4bba-8827-4555e78ba407 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.503807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c4fbc-9282-4bba-8827-4555e78ba407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:46.504094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c4fbc-9282-4bba-8827-4555e78ba407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:46.505486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c4fbc-9282-4bba-8827-4555e78ba407 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:46.517187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 446/891] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:46.518756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfff88-7a52-4f94-8fbd-fafcfab88ddd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.519041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfff88-7a52-4f94-8fbd-fafcfab88ddd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.519307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfff88-7a52-4f94-8fbd-fafcfab88ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.536210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f4dfff88-7a52-4f94-8fbd-fafcfab88ddd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:46.537422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f4dfff88-7a52-4f94-8fbd-fafcfab88ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:46.542254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5552b86-49a8-4e4d-9015-a58184fd96c0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:46.579269 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5ee45-6d29-44ac-a9f4-0e4c68dad429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:46.580160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5ee45-6d29-44ac-a9f4-0e4c68dad429 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.580761 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5ee45-6d29-44ac-a9f4-0e4c68dad429 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.581205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5ee45-6d29-44ac-a9f4-0e4c68dad429 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.582174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5ee45-6d29-44ac-a9f4-0e4c68dad429 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:46.582645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5ee45-6d29-44ac-a9f4-0e4c68dad429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:46.584158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5ee45-6d29-44ac-a9f4-0e4c68dad429 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:46.592440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 446/892] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:46.592440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5ee45-6d29-44ac-a9f4-0e4c68dad429 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:46.604567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:46.620601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 447/893] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:46.620601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfff88-7a52-4f94-8fbd-fafcfab88ddd None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:46.658524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:46.704675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5ee45-6d29-44ac-a9f4-0e4c68dad429 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:46.729688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfff88-7a52-4f94-8fbd-fafcfab88ddd None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:46.740964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27b981-33f2-4189-8200-719d321137e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:46.741780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27b981-33f2-4189-8200-719d321137e4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.742050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27b981-33f2-4189-8200-719d321137e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.742368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27b981-33f2-4189-8200-719d321137e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.749718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27b981-33f2-4189-8200-719d321137e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:46.751265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27b981-33f2-4189-8200-719d321137e4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1c27b981-33f2-4189-8200-719d321137e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:46.763097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:46.764005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 447/894] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:46.765342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27b981-33f2-4189-8200-719d321137e4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:46.769337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76c499-bfe2-44e1-b01b-84525c7668e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:46.769949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76c499-bfe2-44e1-b01b-84525c7668e3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.770227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76c499-bfe2-44e1-b01b-84525c7668e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.770497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76c499-bfe2-44e1-b01b-84525c7668e3 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.771126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76c499-bfe2-44e1-b01b-84525c7668e3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:46.771423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76c499-bfe2-44e1-b01b-84525c7668e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:46.772893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76c499-bfe2-44e1-b01b-84525c7668e3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:46.778785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 448/895] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:46.779569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76c499-bfe2-44e1-b01b-84525c7668e3 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:46.854751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:46.877831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27b981-33f2-4189-8200-719d321137e4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:46.893967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76c499-bfe2-44e1-b01b-84525c7668e3 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:46.895643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:46.914118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f53926-bbf1-40a1-8d42-63a7021c80b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:46.914746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f53926-bbf1-40a1-8d42-63a7021c80b1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.915027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f53926-bbf1-40a1-8d42-63a7021c80b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.915275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f53926-bbf1-40a1-8d42-63a7021c80b1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.915862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f53926-bbf1-40a1-8d42-63a7021c80b1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:46.917243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f53926-bbf1-40a1-8d42-63a7021c80b1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c1f53926-bbf1-40a1-8d42-63a7021c80b1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:46.926980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:46.928104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:46.929298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:46.931360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 448/896] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:46.932264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f53926-bbf1-40a1-8d42-63a7021c80b1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:46.938683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea72d0a-24cd-4f54-adf5-70b9823e3f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:46.939298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea72d0a-24cd-4f54-adf5-70b9823e3f41 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:46.939572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea72d0a-24cd-4f54-adf5-70b9823e3f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:46.939841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea72d0a-24cd-4f54-adf5-70b9823e3f41 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:46.940555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea72d0a-24cd-4f54-adf5-70b9823e3f41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:46.940858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea72d0a-24cd-4f54-adf5-70b9823e3f41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:46.942766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea72d0a-24cd-4f54-adf5-70b9823e3f41 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:46.964050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 449/897] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:46.964050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea72d0a-24cd-4f54-adf5-70b9823e3f41 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:47.036880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f53926-bbf1-40a1-8d42-63a7021c80b1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:47.073093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea72d0a-24cd-4f54-adf5-70b9823e3f41 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:47.074812 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e820c-8d2c-4dd6-92b7-f9c8c078b4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:47.075428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e820c-8d2c-4dd6-92b7-f9c8c078b4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:47.075714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e820c-8d2c-4dd6-92b7-f9c8c078b4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:47.075963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e820c-8d2c-4dd6-92b7-f9c8c078b4a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:47.076711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e820c-8d2c-4dd6-92b7-f9c8c078b4a7 None admin] RBAC: Adding query filter params (interface=public, service_id=cf4d5bd4e7cf46f8ae0fe893a40b277e, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:47.077057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e820c-8d2c-4dd6-92b7-f9c8c078b4a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:47.078736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e820c-8d2c-4dd6-92b7-f9c8c078b4a7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:47.088884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 449/898] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/endpoints?service_id=cf4d5bd4e7cf46f8ae0fe893a40b277e&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:47.088884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e820c-8d2c-4dd6-92b7-f9c8c078b4a7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:47.105819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:47.120002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156ede0-49d1-43ee-a672-cb78fa93f029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:47.120664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156ede0-49d1-43ee-a672-cb78fa93f029 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:47.120930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156ede0-49d1-43ee-a672-cb78fa93f029 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:47.121196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156ede0-49d1-43ee-a672-cb78fa93f029 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:47.121820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156ede0-49d1-43ee-a672-cb78fa93f029 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:47.122115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156ede0-49d1-43ee-a672-cb78fa93f029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:47.123466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156ede0-49d1-43ee-a672-cb78fa93f029 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:47.128947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 450/899] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:51:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:47.129958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156ede0-49d1-43ee-a672-cb78fa93f029 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:47.139870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:47.204354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e820c-8d2c-4dd6-92b7-f9c8c078b4a7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:47.237366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156ede0-49d1-43ee-a672-cb78fa93f029 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:47.242550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b9ca9-0fa3-4e46-99e8-20b8ae7881e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:47.243152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b9ca9-0fa3-4e46-99e8-20b8ae7881e3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:47.243421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b9ca9-0fa3-4e46-99e8-20b8ae7881e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:47.243679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b9ca9-0fa3-4e46-99e8-20b8ae7881e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:47.244397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b9ca9-0fa3-4e46-99e8-20b8ae7881e3 None admin] RBAC: Adding query filter params (role.id=b81578bacade480d94c41559bc214637, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=4f0ca85ea7974962b8115428563aef8f) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:47.244724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b9ca9-0fa3-4e46-99e8-20b8ae7881e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:47.246254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b9ca9-0fa3-4e46-99e8-20b8ae7881e3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:47.252435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 450/900] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:47 2022] GET /identity/v3/role_assignments?user.id=4f0ca85ea7974962b8115428563aef8f&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=b81578bacade480d94c41559bc214637 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:47.253405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b9ca9-0fa3-4e46-99e8-20b8ae7881e3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:47.293025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd3284-c588-4d69-b446-47c3908f700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:47.294792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd3284-c588-4d69-b446-47c3908f700d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:47.295265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd3284-c588-4d69-b446-47c3908f700d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:47.296002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd3284-c588-4d69-b446-47c3908f700d None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:47.296885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd3284-c588-4d69-b446-47c3908f700d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:47.298824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd3284-c588-4d69-b446-47c3908f700d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:47.317132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 451/901] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 17:51:47 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:51:47.392832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b9ca9-0fa3-4e46-99e8-20b8ae7881e3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:47.414594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:47.450530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e808c-a202-4c3f-9339-fdd8b9eba97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:47.451541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e808c-a202-4c3f-9339-fdd8b9eba97f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:47.451917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e808c-a202-4c3f-9339-fdd8b9eba97f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:47.452328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e808c-a202-4c3f-9339-fdd8b9eba97f None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:47.453128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e808c-a202-4c3f-9339-fdd8b9eba97f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:47.453784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e808c-a202-4c3f-9339-fdd8b9eba97f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:47.455292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e808c-a202-4c3f-9339-fdd8b9eba97f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:47.461205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 451/902] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:51:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:47.607788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d077e3-4860-48a9-b0df-568a3ce8fc73 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:47.608309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d077e3-4860-48a9-b0df-568a3ce8fc73 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:47.608797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d077e3-4860-48a9-b0df-568a3ce8fc73 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:47.610557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 452/903] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:47.616225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a96cf0-5f90-4127-b6cb-a083500f2bae None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:47.616523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a96cf0-5f90-4127-b6cb-a083500f2bae None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:47.616801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a96cf0-5f90-4127-b6cb-a083500f2bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:47.636405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f0a96cf0-5f90-4127-b6cb-a083500f2bae None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:47.637653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f0a96cf0-5f90-4127-b6cb-a083500f2bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:47.718230 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 452/904] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:47.744955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d077e3-4860-48a9-b0df-568a3ce8fc73 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:47.803720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:47.862797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d077e3-4860-48a9-b0df-568a3ce8fc73 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:47.903151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dbd31-da8e-4c87-a1a4-f02aac128a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:47.904148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dbd31-da8e-4c87-a1a4-f02aac128a14 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:47.904666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dbd31-da8e-4c87-a1a4-f02aac128a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:47.905163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dbd31-da8e-4c87-a1a4-f02aac128a14 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:47.905941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dbd31-da8e-4c87-a1a4-f02aac128a14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:47.907611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dbd31-da8e-4c87-a1a4-f02aac128a14 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c40dbd31-da8e-4c87-a1a4-f02aac128a14 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:47.917258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:47.918274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 17:51:47.919202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:47.922159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 453/905] 10.222.0.50 () {56 vars in 1132 bytes} [Mon Aug 8 17:51:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:47.929214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a96cf0-5f90-4127-b6cb-a083500f2bae None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:48.035735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a96cf0-5f90-4127-b6cb-a083500f2bae None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:48.038026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:48.077628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e7409-c3f9-4f3a-86aa-4825760a7697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:48.077628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e7409-c3f9-4f3a-86aa-4825760a7697 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:48.077628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e7409-c3f9-4f3a-86aa-4825760a7697 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:48.077628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e7409-c3f9-4f3a-86aa-4825760a7697 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:48.078054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e7409-c3f9-4f3a-86aa-4825760a7697 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:48.078322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e7409-c3f9-4f3a-86aa-4825760a7697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:48.080116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e7409-c3f9-4f3a-86aa-4825760a7697 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:48.085771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 453/906] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:48.091612 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dbd31-da8e-4c87-a1a4-f02aac128a14 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:48.196452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dbd31-da8e-4c87-a1a4-f02aac128a14 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:48.211663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:48.231898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f3111-6a56-4192-8304-c8dd45a734e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:48.232493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f3111-6a56-4192-8304-c8dd45a734e2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:48.232793 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f3111-6a56-4192-8304-c8dd45a734e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:48.233056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f3111-6a56-4192-8304-c8dd45a734e2 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:48.233679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f3111-6a56-4192-8304-c8dd45a734e2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:48.233971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f3111-6a56-4192-8304-c8dd45a734e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:48.235333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f3111-6a56-4192-8304-c8dd45a734e2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:48.244606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 454/907] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:48.247002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e7409-c3f9-4f3a-86aa-4825760a7697 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:48.352678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e7409-c3f9-4f3a-86aa-4825760a7697 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:48.388487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b609a-34c0-457b-9827-030e360173be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:48.389835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b609a-34c0-457b-9827-030e360173be None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:48.390135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b609a-34c0-457b-9827-030e360173be None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:48.390406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b609a-34c0-457b-9827-030e360173be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:48.391099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b609a-34c0-457b-9827-030e360173be None admin] RBAC: Adding query filter params (interface=public, service_id=3ee60519e8074eb9b9788b4e263d5486, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:48.391403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b609a-34c0-457b-9827-030e360173be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:48.392785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b609a-34c0-457b-9827-030e360173be None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:48.398427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 454/908] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 17:51:48 2022] GET /identity/v3/endpoints?service_id=3ee60519e8074eb9b9788b4e263d5486&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:48.403847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f3111-6a56-4192-8304-c8dd45a734e2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:48.520826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f3111-6a56-4192-8304-c8dd45a734e2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:48.558835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187bfcf-a965-4bae-a3ff-e64c47c67024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:48.559441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187bfcf-a965-4bae-a3ff-e64c47c67024 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:48.559715 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187bfcf-a965-4bae-a3ff-e64c47c67024 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:48.559983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187bfcf-a965-4bae-a3ff-e64c47c67024 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:48.560610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187bfcf-a965-4bae-a3ff-e64c47c67024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:48.561979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187bfcf-a965-4bae-a3ff-e64c47c67024 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:48.568346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 455/909] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 17:51:48 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:51:49.419011 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e9791-fa33-4cb2-87b0-5177c4ef2e92 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.419535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e9791-fa33-4cb2-87b0-5177c4ef2e92 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.419720 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e9791-fa33-4cb2-87b0-5177c4ef2e92 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.421276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 455/910] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:49.426716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf18578-df01-48ca-afa0-54d5554e8083 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.427113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf18578-df01-48ca-afa0-54d5554e8083 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.427295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf18578-df01-48ca-afa0-54d5554e8083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.435093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499bdf4-1614-4fba-93d5-838490950595 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.435367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499bdf4-1614-4fba-93d5-838490950595 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.435633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499bdf4-1614-4fba-93d5-838490950595 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.437236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 456/911] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:49.442154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec187760-86a3-422a-84e8-56b1079108ca None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.442435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec187760-86a3-422a-84e8-56b1079108ca None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.442704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec187760-86a3-422a-84e8-56b1079108ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.446250 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-cdf18578-df01-48ca-afa0-54d5554e8083 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:49.447443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cdf18578-df01-48ca-afa0-54d5554e8083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:49.461772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ec187760-86a3-422a-84e8-56b1079108ca None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:49.463206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ec187760-86a3-422a-84e8-56b1079108ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:49.491210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:49.493194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:49.531621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 456/912] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:49.542717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 457/913] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:49.556258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf18578-df01-48ca-afa0-54d5554e8083 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:49.565726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec187760-86a3-422a-84e8-56b1079108ca None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:49.663061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf18578-df01-48ca-afa0-54d5554e8083 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:49.678415 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec187760-86a3-422a-84e8-56b1079108ca None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:49.699287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48accab0-c4ed-4275-a915-7032b1a144ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:49.699913 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48accab0-c4ed-4275-a915-7032b1a144ab None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.700189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48accab0-c4ed-4275-a915-7032b1a144ab None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.700460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48accab0-c4ed-4275-a915-7032b1a144ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.713528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48accab0-c4ed-4275-a915-7032b1a144ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-48accab0-c4ed-4275-a915-7032b1a144ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:49.718382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:49.719541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:49.720568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 457/914] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 17:51:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:49.725483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea9358-041e-46a3-a4aa-82fffc68782f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.725759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea9358-041e-46a3-a4aa-82fffc68782f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.726026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea9358-041e-46a3-a4aa-82fffc68782f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.726659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb875d2-f45c-4ff0-8495-b8af7fa2e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:49.727278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 458/915] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:49.727654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb875d2-f45c-4ff0-8495-b8af7fa2e97d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.728053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb875d2-f45c-4ff0-8495-b8af7fa2e97d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.728401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea9358-041e-46a3-a4aa-82fffc68782f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:49.728595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb875d2-f45c-4ff0-8495-b8af7fa2e97d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.729418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb875d2-f45c-4ff0-8495-b8af7fa2e97d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:49.730904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb875d2-f45c-4ff0-8495-b8af7fa2e97d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-feb875d2-f45c-4ff0-8495-b8af7fa2e97d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:49.742514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:49.743730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 17:51:49.744794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:49.747674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 458/916] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:49.748598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bb3fa-e170-4513-86ca-c4306f41ad28 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.748877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bb3fa-e170-4513-86ca-c4306f41ad28 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.749148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bb3fa-e170-4513-86ca-c4306f41ad28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.765519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:49.767739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-782bb3fa-e170-4513-86ca-c4306f41ad28 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:49.768957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-782bb3fa-e170-4513-86ca-c4306f41ad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:49.793718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:49.840557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 459/917] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:49.841394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bb3fa-e170-4513-86ca-c4306f41ad28 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:49.855453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea9358-041e-46a3-a4aa-82fffc68782f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:49.896218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560c8c0-c98a-486c-a712-c68b3652e18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:49.897125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560c8c0-c98a-486c-a712-c68b3652e18f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.897398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560c8c0-c98a-486c-a712-c68b3652e18f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.897852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560c8c0-c98a-486c-a712-c68b3652e18f None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.898870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560c8c0-c98a-486c-a712-c68b3652e18f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:49.899175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560c8c0-c98a-486c-a712-c68b3652e18f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:49.900550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560c8c0-c98a-486c-a712-c68b3652e18f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:49.913787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 459/918] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 17:51:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:49.913787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560c8c0-c98a-486c-a712-c68b3652e18f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:49.949351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bb3fa-e170-4513-86ca-c4306f41ad28 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:49.985533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fa691-2097-4d15-83a2-76c0e2cf32fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:49.986420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fa691-2097-4d15-83a2-76c0e2cf32fe None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:49.986705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fa691-2097-4d15-83a2-76c0e2cf32fe None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:49.987027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fa691-2097-4d15-83a2-76c0e2cf32fe None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:49.987657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fa691-2097-4d15-83a2-76c0e2cf32fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:49.987953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fa691-2097-4d15-83a2-76c0e2cf32fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:49.989344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fa691-2097-4d15-83a2-76c0e2cf32fe None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:49.994056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 460/919] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:49.994863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fa691-2097-4d15-83a2-76c0e2cf32fe None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:50.018265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:50.018589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:50.025413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560c8c0-c98a-486c-a712-c68b3652e18f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:50.061437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8b9c-9955-4ba5-a358-63ea754c7904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:50.062037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8b9c-9955-4ba5-a358-63ea754c7904 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.062316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8b9c-9955-4ba5-a358-63ea754c7904 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.062585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8b9c-9955-4ba5-a358-63ea754c7904 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.063230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8b9c-9955-4ba5-a358-63ea754c7904 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:50.064624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8b9c-9955-4ba5-a358-63ea754c7904 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:50.100840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fa691-2097-4d15-83a2-76c0e2cf32fe None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:50.134644 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 460/920] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:51:49 2022] POST /identity/v3/services => generated 241 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:50.135608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8b9c-9955-4ba5-a358-63ea754c7904 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:50.150168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9290ef1-e19c-43e4-a408-d2334f46f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:50.150928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9290ef1-e19c-43e4-a408-d2334f46f803 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.151277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9290ef1-e19c-43e4-a408-d2334f46f803 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.151652 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9290ef1-e19c-43e4-a408-d2334f46f803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.165613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9290ef1-e19c-43e4-a408-d2334f46f803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:50.167947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9290ef1-e19c-43e4-a408-d2334f46f803 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d9290ef1-e19c-43e4-a408-d2334f46f803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:50.186516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:50.187551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:50.188431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:50.188431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:50.188431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:50.188431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 461/921] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:50.192946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9290ef1-e19c-43e4-a408-d2334f46f803 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:50.274950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:50.289058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8b9c-9955-4ba5-a358-63ea754c7904 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:50.316186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9290ef1-e19c-43e4-a408-d2334f46f803 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:50.335070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d610f5-8ada-41dc-89fc-93645f71b134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:50.335070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d610f5-8ada-41dc-89fc-93645f71b134 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.335473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d610f5-8ada-41dc-89fc-93645f71b134 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.335634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d610f5-8ada-41dc-89fc-93645f71b134 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.336277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d610f5-8ada-41dc-89fc-93645f71b134 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:50.337933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d610f5-8ada-41dc-89fc-93645f71b134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:50.338822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d610f5-8ada-41dc-89fc-93645f71b134 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:50.348840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 461/922] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 17:51:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:50.350480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d610f5-8ada-41dc-89fc-93645f71b134 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:50.369371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49351a2-ccf6-4f86-83e0-b90b20263714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:50.371057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49351a2-ccf6-4f86-83e0-b90b20263714 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.371505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49351a2-ccf6-4f86-83e0-b90b20263714 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.372095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49351a2-ccf6-4f86-83e0-b90b20263714 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.373252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49351a2-ccf6-4f86-83e0-b90b20263714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:50.373746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49351a2-ccf6-4f86-83e0-b90b20263714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:50.375295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49351a2-ccf6-4f86-83e0-b90b20263714 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:50.388889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:50.392833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 462/923] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:50.404656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49351a2-ccf6-4f86-83e0-b90b20263714 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:50.463516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d610f5-8ada-41dc-89fc-93645f71b134 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:50.501479 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72220b8a-8d8c-4a97-b870-e559baf44d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:50.502182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72220b8a-8d8c-4a97-b870-e559baf44d75 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.502551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72220b8a-8d8c-4a97-b870-e559baf44d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.502894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72220b8a-8d8c-4a97-b870-e559baf44d75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.503620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72220b8a-8d8c-4a97-b870-e559baf44d75 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:50.505112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72220b8a-8d8c-4a97-b870-e559baf44d75 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:50.513454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49351a2-ccf6-4f86-83e0-b90b20263714 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:50.542187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:50.550211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccca85a-7f90-494e-a1a3-e7e4452343d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:50.550843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccca85a-7f90-494e-a1a3-e7e4452343d9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.551118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccca85a-7f90-494e-a1a3-e7e4452343d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.551386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccca85a-7f90-494e-a1a3-e7e4452343d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.559151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccca85a-7f90-494e-a1a3-e7e4452343d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:50.560595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccca85a-7f90-494e-a1a3-e7e4452343d9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-fccca85a-7f90-494e-a1a3-e7e4452343d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:50.572665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:50.581241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 463/924] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:51:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:50.582191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccca85a-7f90-494e-a1a3-e7e4452343d9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:50.587589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 462/925] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 17:51:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:51:50.629187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bb165-270b-4509-90a3-763f6167b259 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.630347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bb165-270b-4509-90a3-763f6167b259 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.630844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bb165-270b-4509-90a3-763f6167b259 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.632909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 463/926] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:50.642234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480905d-7f26-4180-95b3-5584485707b9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.642234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480905d-7f26-4180-95b3-5584485707b9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.642234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480905d-7f26-4180-95b3-5584485707b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.663355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f480905d-7f26-4180-95b3-5584485707b9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:50.664927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f480905d-7f26-4180-95b3-5584485707b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:50.712126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccca85a-7f90-494e-a1a3-e7e4452343d9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:50.758098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa530313-11b0-4416-9c29-3cfc9a7f7e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:50.758779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:50.759927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa530313-11b0-4416-9c29-3cfc9a7f7e28 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.760380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa530313-11b0-4416-9c29-3cfc9a7f7e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.760838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa530313-11b0-4416-9c29-3cfc9a7f7e28 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.761704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa530313-11b0-4416-9c29-3cfc9a7f7e28 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:50.762144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa530313-11b0-4416-9c29-3cfc9a7f7e28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:50.763766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa530313-11b0-4416-9c29-3cfc9a7f7e28 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:50.764801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 464/927] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:50.786854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 464/928] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 17:51:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:50.790202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480905d-7f26-4180-95b3-5584485707b9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:50.792789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa530313-11b0-4416-9c29-3cfc9a7f7e28 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:50.843659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:50.911827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480905d-7f26-4180-95b3-5584485707b9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:50.921922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa530313-11b0-4416-9c29-3cfc9a7f7e28 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:50.948168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f51d-3974-4a2b-bc9a-f4845210174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:50.949187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f51d-3974-4a2b-bc9a-f4845210174c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.949660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f51d-3974-4a2b-bc9a-f4845210174c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.950151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f51d-3974-4a2b-bc9a-f4845210174c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.950917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f51d-3974-4a2b-bc9a-f4845210174c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:50.952466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f51d-3974-4a2b-bc9a-f4845210174c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-6464f51d-3974-4a2b-bc9a-f4845210174c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:50.961798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:50.962751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 17:51:50.963641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:50.965716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 465/929] 10.222.0.50 () {56 vars in 1132 bytes} [Mon Aug 8 17:51:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:50.971867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f51d-3974-4a2b-bc9a-f4845210174c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:50.972867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c0f68-8573-420f-8e8a-9ad861b83b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:50.973499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c0f68-8573-420f-8e8a-9ad861b83b0e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:50.973763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c0f68-8573-420f-8e8a-9ad861b83b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:50.974020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c0f68-8573-420f-8e8a-9ad861b83b0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:50.987959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c0f68-8573-420f-8e8a-9ad861b83b0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:50.989719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c0f68-8573-420f-8e8a-9ad861b83b0e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-fd6c0f68-8573-420f-8e8a-9ad861b83b0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:51.009866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:51.010894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:51.011800 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:51.011800 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:51.011800 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:51.014070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 465/930] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:51.020621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c0f68-8573-420f-8e8a-9ad861b83b0e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:51.047945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:51.097332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f51d-3974-4a2b-bc9a-f4845210174c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:51.122856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:51.130606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c0f68-8573-420f-8e8a-9ad861b83b0e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:51.139658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a8867-5bab-40da-af47-5fa0256a308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:51.140258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8867-5bab-40da-af47-5fa0256a308e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:51.140544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8867-5bab-40da-af47-5fa0256a308e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:51.140814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8867-5bab-40da-af47-5fa0256a308e None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:51.141870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a8867-5bab-40da-af47-5fa0256a308e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:51.142172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a8867-5bab-40da-af47-5fa0256a308e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:51.143639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a8867-5bab-40da-af47-5fa0256a308e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:51.149796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 466/931] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:51.155478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a8867-5bab-40da-af47-5fa0256a308e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:51.172009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57bec-e4f7-416e-942b-f524ee1ff5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:51.172649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57bec-e4f7-416e-942b-f524ee1ff5dc None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:51.172926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57bec-e4f7-416e-942b-f524ee1ff5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:51.173194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57bec-e4f7-416e-942b-f524ee1ff5dc None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:51.173845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff57bec-e4f7-416e-942b-f524ee1ff5dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:51.174142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff57bec-e4f7-416e-942b-f524ee1ff5dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:51.175727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff57bec-e4f7-416e-942b-f524ee1ff5dc None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:51.192804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 466/932] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:51.193699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57bec-e4f7-416e-942b-f524ee1ff5dc None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:51.277611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a8867-5bab-40da-af47-5fa0256a308e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:51.302724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57bec-e4f7-416e-942b-f524ee1ff5dc None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:51.317234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:51.326064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f75440-2e9a-4f42-82b6-b13b0d1a3dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:51.331492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f75440-2e9a-4f42-82b6-b13b0d1a3dee None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:51.331492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f75440-2e9a-4f42-82b6-b13b0d1a3dee None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:51.331492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f75440-2e9a-4f42-82b6-b13b0d1a3dee None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:51.331492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f75440-2e9a-4f42-82b6-b13b0d1a3dee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:51.331492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f75440-2e9a-4f42-82b6-b13b0d1a3dee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:51.331492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f75440-2e9a-4f42-82b6-b13b0d1a3dee None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:51.340317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 467/933] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:51.341331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f75440-2e9a-4f42-82b6-b13b0d1a3dee None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:51.354393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834ccb-4dca-4526-b65b-42d810741d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:51.356535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834ccb-4dca-4526-b65b-42d810741d91 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:51.357198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834ccb-4dca-4526-b65b-42d810741d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:51.357750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834ccb-4dca-4526-b65b-42d810741d91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:51.367851 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad834ccb-4dca-4526-b65b-42d810741d91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:51.370106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad834ccb-4dca-4526-b65b-42d810741d91 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ad834ccb-4dca-4526-b65b-42d810741d91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:51.387423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:51.388611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:51.389748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:51.389748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:51.389748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:51.389748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:51.389748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:51.389748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:51.389748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:51.389748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:51.397002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 467/934] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:51.397614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834ccb-4dca-4526-b65b-42d810741d91 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:51.420596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:51.463497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f75440-2e9a-4f42-82b6-b13b0d1a3dee None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:51.500858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89368889-dc5f-4aa1-83c2-4a26aeefebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:51.501554 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89368889-dc5f-4aa1-83c2-4a26aeefebb8 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:51.502015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89368889-dc5f-4aa1-83c2-4a26aeefebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:51.502369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89368889-dc5f-4aa1-83c2-4a26aeefebb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:51.503129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89368889-dc5f-4aa1-83c2-4a26aeefebb8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:51.504805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89368889-dc5f-4aa1-83c2-4a26aeefebb8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:51.529826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834ccb-4dca-4526-b65b-42d810741d91 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:51.565056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0b4ef-a352-4b77-9f6e-7bcced8eadd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:51.565596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0b4ef-a352-4b77-9f6e-7bcced8eadd5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:51.565872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0b4ef-a352-4b77-9f6e-7bcced8eadd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:51.566140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0b4ef-a352-4b77-9f6e-7bcced8eadd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:51.566845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0b4ef-a352-4b77-9f6e-7bcced8eadd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:51.567133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0b4ef-a352-4b77-9f6e-7bcced8eadd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:51.568580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0b4ef-a352-4b77-9f6e-7bcced8eadd5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:51.585215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 468/935] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 17:51:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:51.589260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 468/936] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:51.595650 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89368889-dc5f-4aa1-83c2-4a26aeefebb8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:51.653280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:51.717744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89368889-dc5f-4aa1-83c2-4a26aeefebb8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:51.761864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef70053-1be5-4f5d-a04e-d93c11cbd34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:51.762538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef70053-1be5-4f5d-a04e-d93c11cbd34f None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:51.762808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef70053-1be5-4f5d-a04e-d93c11cbd34f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:51.763071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef70053-1be5-4f5d-a04e-d93c11cbd34f None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/4f0ca85ea7974962b8115428563aef8f/roles/b81578bacade480d94c41559bc214637` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:51.765748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef70053-1be5-4f5d-a04e-d93c11cbd34f None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=4f0ca85ea7974962b8115428563aef8f, role_id=b81578bacade480d94c41559bc214637)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:51.767196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef70053-1be5-4f5d-a04e-d93c11cbd34f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:51.820829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 469/937] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:51:51 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/4f0ca85ea7974962b8115428563aef8f/roles/b81578bacade480d94c41559bc214637 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:51:52.267238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8585ed2-39af-41d2-87e7-3b54c1303a93 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:52.267433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8585ed2-39af-41d2-87e7-3b54c1303a93 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:52.267696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8585ed2-39af-41d2-87e7-3b54c1303a93 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:52.269305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 469/938] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:52.275224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568213d-aedc-4d87-9e59-c932dee86087 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:52.275513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568213d-aedc-4d87-9e59-c932dee86087 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:52.275781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568213d-aedc-4d87-9e59-c932dee86087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:52.294596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9568213d-aedc-4d87-9e59-c932dee86087 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:52.295787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9568213d-aedc-4d87-9e59-c932dee86087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:52.367953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 470/939] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:52.390862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8585ed2-39af-41d2-87e7-3b54c1303a93 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:52.471491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:52.498731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8585ed2-39af-41d2-87e7-3b54c1303a93 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:52.534848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17497c-bd50-4881-a7ad-f3c8705e4f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:52.535436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17497c-bd50-4881-a7ad-f3c8705e4f3d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:52.535694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17497c-bd50-4881-a7ad-f3c8705e4f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:52.535964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17497c-bd50-4881-a7ad-f3c8705e4f3d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:52.536607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17497c-bd50-4881-a7ad-f3c8705e4f3d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:52.537965 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17497c-bd50-4881-a7ad-f3c8705e4f3d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-0b17497c-bd50-4881-a7ad-f3c8705e4f3d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:52.546835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:52.547824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:52.548714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:52.552622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 470/940] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:52.554349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568213d-aedc-4d87-9e59-c932dee86087 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:52.590621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:52.660083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568213d-aedc-4d87-9e59-c932dee86087 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:52.700422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5d86c-60dc-4428-a97b-0ff7616b889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:52.701280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5d86c-60dc-4428-a97b-0ff7616b889d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:52.701728 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5d86c-60dc-4428-a97b-0ff7616b889d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:52.702133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5d86c-60dc-4428-a97b-0ff7616b889d None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:52.702929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5d86c-60dc-4428-a97b-0ff7616b889d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:52.703361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5d86c-60dc-4428-a97b-0ff7616b889d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:52.705196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5d86c-60dc-4428-a97b-0ff7616b889d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:52.710472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 471/941] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:51:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:52.716309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17497c-bd50-4881-a7ad-f3c8705e4f3d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:52.728223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e6ad9-8f4c-4d48-ac51-f4dc91bd3eee None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:52.728631 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e6ad9-8f4c-4d48-ac51-f4dc91bd3eee None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:52.728804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e6ad9-8f4c-4d48-ac51-f4dc91bd3eee None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:52.730534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 472/942] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:52.735403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce5e4c-37ea-45b9-aea0-18ea166901d5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:52.735632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce5e4c-37ea-45b9-aea0-18ea166901d5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:52.735898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce5e4c-37ea-45b9-aea0-18ea166901d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:52.755157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9cce5e4c-37ea-45b9-aea0-18ea166901d5 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:52.756338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9cce5e4c-37ea-45b9-aea0-18ea166901d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:52.846373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17497c-bd50-4881-a7ad-f3c8705e4f3d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:52.850415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 473/943] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:51:52.874183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce5e4c-37ea-45b9-aea0-18ea166901d5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:52.887801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f176bd1-3a4c-4aed-b054-571ff785b60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:52.888856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f176bd1-3a4c-4aed-b054-571ff785b60f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:52.889325 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f176bd1-3a4c-4aed-b054-571ff785b60f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:52.889783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f176bd1-3a4c-4aed-b054-571ff785b60f None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:52.890616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f176bd1-3a4c-4aed-b054-571ff785b60f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:52.891260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f176bd1-3a4c-4aed-b054-571ff785b60f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:52.892997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f176bd1-3a4c-4aed-b054-571ff785b60f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:52.898620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 471/944] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:51:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:52.904131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f176bd1-3a4c-4aed-b054-571ff785b60f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:52.909609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:52.910930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:52.992805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce5e4c-37ea-45b9-aea0-18ea166901d5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:53.039627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f176bd1-3a4c-4aed-b054-571ff785b60f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:53.048469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:53.049805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.050232 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.050647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.051953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:53.054366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:53.058703 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WARNING keystone.server.flask.common [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 17:51:53.080862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a40a57-94ef-4e2b-8b5c-8f1b29389fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:53.081621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a40a57-94ef-4e2b-8b5c-8f1b29389fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.081959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a40a57-94ef-4e2b-8b5c-8f1b29389fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.082379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a40a57-94ef-4e2b-8b5c-8f1b29389fc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.083137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a40a57-94ef-4e2b-8b5c-8f1b29389fc6 None admin] RBAC: Adding query filter params (interface=public, service_id=4740c5a94c6341b38753f0a266315ae1, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:53.083535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a40a57-94ef-4e2b-8b5c-8f1b29389fc6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:53.085040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a40a57-94ef-4e2b-8b5c-8f1b29389fc6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:53.092552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 472/945] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 17:51:52 2022] GET /identity/v3/endpoints?service_id=4740c5a94c6341b38753f0a266315ae1&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:53.095808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a40a57-94ef-4e2b-8b5c-8f1b29389fc6 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:53.122662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.sql.core [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 17:51:53.122662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 17:51:53.122662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [parameters: {'id': '883a636e71814b0daa77bf2a9bd1f1ae', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 17:51:53.122662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80402) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 883a636e71814b0daa77bf2a9bd1f1ae.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 883a636e71814b0daa77bf2a9bd1f1ae. Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application conn.query(q) Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 17:51:53.138953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result.read() Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 17:51:53.139839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application next(self.gen) Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application yield resource Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application next(self.gen) Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise self.value Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application session.commit() Aug 08 17:51:53.141299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.session.flush() Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application compat.raise_( Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 17:51:53.142214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 17:51:53.143117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application conn.query(q) Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result.read() Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 17:51:53.143914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [parameters: {'id': '883a636e71814b0daa77bf2a9bd1f1ae', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:53.145935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:53.147861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 883a636e71814b0daa77bf2a9bd1f1ae. Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 474/946] 10.222.0.50 () {60 vars in 1140 bytes} [Mon Aug 8 17:51:52 2022] POST /identity/v3/projects => generated 262 bytes in 272 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:53.151207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:53.190495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:53.234856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a40a57-94ef-4e2b-8b5c-8f1b29389fc6 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:53.243921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:53.262360 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c21a43-c259-4b15-aba3-857536b149fe None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:53.277178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14061c31-d171-4cba-be04-1012c4b208d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:53.277765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14061c31-d171-4cba-be04-1012c4b208d4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.278045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14061c31-d171-4cba-be04-1012c4b208d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.278430 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14061c31-d171-4cba-be04-1012c4b208d4 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.279037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14061c31-d171-4cba-be04-1012c4b208d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:53.280590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14061c31-d171-4cba-be04-1012c4b208d4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:53.288537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 473/947] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 17:51:53 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:51:53.315329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe951a-bbb7-4749-a72c-5fcb6a4d4409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:53.316352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe951a-bbb7-4749-a72c-5fcb6a4d4409 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.317071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe951a-bbb7-4749-a72c-5fcb6a4d4409 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.317687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe951a-bbb7-4749-a72c-5fcb6a4d4409 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.326233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe951a-bbb7-4749-a72c-5fcb6a4d4409 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:53.328285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe951a-bbb7-4749-a72c-5fcb6a4d4409 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-ebbe951a-bbb7-4749-a72c-5fcb6a4d4409 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:53.342194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:53.343219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:53.344124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:53.344124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:53.344124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:53.344124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:53.344124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:53.344124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:53.344124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:53.344124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:53.346834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 475/948] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:53.352320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14061c31-d171-4cba-be04-1012c4b208d4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:53.468145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14061c31-d171-4cba-be04-1012c4b208d4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:53.499129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:53.505230 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fca40-4ccf-4e02-8b8a-af235df2f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:53.505904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fca40-4ccf-4e02-8b8a-af235df2f4ea None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.506242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fca40-4ccf-4e02-8b8a-af235df2f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.506631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fca40-4ccf-4e02-8b8a-af235df2f4ea None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.507485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fca40-4ccf-4e02-8b8a-af235df2f4ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:53.507847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fca40-4ccf-4e02-8b8a-af235df2f4ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:53.509436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fca40-4ccf-4e02-8b8a-af235df2f4ea None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:53.527941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 474/949] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:53.719516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f0944-b0e3-4f72-be0b-cc9e9f959a46 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.719996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f0944-b0e3-4f72-be0b-cc9e9f959a46 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.719996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f0944-b0e3-4f72-be0b-cc9e9f959a46 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.724311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 476/950] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:53.730068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbefd5e-cb89-4baf-8b37-96614cac10f2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.730533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbefd5e-cb89-4baf-8b37-96614cac10f2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.730626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbefd5e-cb89-4baf-8b37-96614cac10f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.749492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8dbefd5e-cb89-4baf-8b37-96614cac10f2 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:53.750718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8dbefd5e-cb89-4baf-8b37-96614cac10f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:53.821074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 475/951] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:53.842980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f0944-b0e3-4f72-be0b-cc9e9f959a46 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:53.954835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39856f-d80a-4167-a10d-674e10da2e97 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.955114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39856f-d80a-4167-a10d-674e10da2e97 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.955386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39856f-d80a-4167-a10d-674e10da2e97 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.955503 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f0944-b0e3-4f72-be0b-cc9e9f959a46 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:53.958114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 476/952] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:53.962244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b6d52-d569-43cc-ba89-413791844ca7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.962529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b6d52-d569-43cc-ba89-413791844ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.962806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b6d52-d569-43cc-ba89-413791844ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.981428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0e6b6d52-d569-43cc-ba89-413791844ca7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:53.982532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0e6b6d52-d569-43cc-ba89-413791844ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:53.996676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03f794-b902-452e-bce7-ac549c33fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:53.997704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03f794-b902-452e-bce7-ac549c33fe01 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:53.998531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03f794-b902-452e-bce7-ac549c33fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:53.998910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03f794-b902-452e-bce7-ac549c33fe01 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:53.999845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03f794-b902-452e-bce7-ac549c33fe01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:54.001566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03f794-b902-452e-bce7-ac549c33fe01 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8a03f794-b902-452e-bce7-ac549c33fe01 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:54.010872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:54.012221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:54.013317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 477/953] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:54.017708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03f794-b902-452e-bce7-ac549c33fe01 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:54.028871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:54.064561 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 477/954] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:54.087749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b6d52-d569-43cc-ba89-413791844ca7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:54.132667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03f794-b902-452e-bce7-ac549c33fe01 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:54.146087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:54.169068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cd7dc-af0f-4ffb-b831-6ec4eec4c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:54.169716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cd7dc-af0f-4ffb-b831-6ec4eec4c58a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:54.170014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cd7dc-af0f-4ffb-b831-6ec4eec4c58a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:54.170272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cd7dc-af0f-4ffb-b831-6ec4eec4c58a None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:54.170888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cd7dc-af0f-4ffb-b831-6ec4eec4c58a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:54.171179 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cd7dc-af0f-4ffb-b831-6ec4eec4c58a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:54.172606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cd7dc-af0f-4ffb-b831-6ec4eec4c58a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:54.182123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 478/955] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:54.182986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cd7dc-af0f-4ffb-b831-6ec4eec4c58a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:54.207589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b6d52-d569-43cc-ba89-413791844ca7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:54.257466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:54.258589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd511d-d942-4f18-b152-a87c0ac53270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:54.259213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd511d-d942-4f18-b152-a87c0ac53270 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:54.259494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd511d-d942-4f18-b152-a87c0ac53270 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:54.259751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd511d-d942-4f18-b152-a87c0ac53270 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:54.278166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd511d-d942-4f18-b152-a87c0ac53270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-d3fd511d-d942-4f18-b152-a87c0ac53270 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:51:54.282996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:54.283951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:54.289902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 478/956] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 17:51:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:54.291573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd511d-d942-4f18-b152-a87c0ac53270 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:54.302780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cd7dc-af0f-4ffb-b831-6ec4eec4c58a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:54.339501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdd29c-3690-4c19-947a-2d0adc0ac473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:54.340368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdd29c-3690-4c19-947a-2d0adc0ac473 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:54.340834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdd29c-3690-4c19-947a-2d0adc0ac473 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:54.341257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdd29c-3690-4c19-947a-2d0adc0ac473 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:54.342016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdd29c-3690-4c19-947a-2d0adc0ac473 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:54.342489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdd29c-3690-4c19-947a-2d0adc0ac473 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:54.344166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdd29c-3690-4c19-947a-2d0adc0ac473 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:54.349230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 479/957] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:54.407902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd511d-d942-4f18-b152-a87c0ac53270 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:54.421227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:54.447794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc86c5b-ce1c-445e-83dc-fd41fc7f6faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:54.448978 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc86c5b-ce1c-445e-83dc-fd41fc7f6faa None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:54.449253 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc86c5b-ce1c-445e-83dc-fd41fc7f6faa None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:54.449521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc86c5b-ce1c-445e-83dc-fd41fc7f6faa None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:54.450202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc86c5b-ce1c-445e-83dc-fd41fc7f6faa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:54.450500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc86c5b-ce1c-445e-83dc-fd41fc7f6faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:54.451876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc86c5b-ce1c-445e-83dc-fd41fc7f6faa None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:54.465675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 479/958] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 17:51:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:54.469643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdd29c-3690-4c19-947a-2d0adc0ac473 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:54.583238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdd29c-3690-4c19-947a-2d0adc0ac473 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:54.619526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69bdf8-e815-4d28-b7dc-d5d451b76a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:54.620218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69bdf8-e815-4d28-b7dc-d5d451b76a11 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:54.620567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69bdf8-e815-4d28-b7dc-d5d451b76a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:54.620879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69bdf8-e815-4d28-b7dc-d5d451b76a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:54.623597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:54.632066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69bdf8-e815-4d28-b7dc-d5d451b76a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:54.633667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69bdf8-e815-4d28-b7dc-d5d451b76a11 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-7d69bdf8-e815-4d28-b7dc-d5d451b76a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:54.649078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:54.650255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:54.651285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:54.651285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:54.651285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:54.651972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 480/959] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:54.657482 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc86c5b-ce1c-445e-83dc-fd41fc7f6faa None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:54.771605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc86c5b-ce1c-445e-83dc-fd41fc7f6faa None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:54.810385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dea3da-8f11-40ec-9282-4e062d63be61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:54.810989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dea3da-8f11-40ec-9282-4e062d63be61 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:54.811264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dea3da-8f11-40ec-9282-4e062d63be61 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:54.811536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dea3da-8f11-40ec-9282-4e062d63be61 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:54.812240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dea3da-8f11-40ec-9282-4e062d63be61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:54.812567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dea3da-8f11-40ec-9282-4e062d63be61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:54.813928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dea3da-8f11-40ec-9282-4e062d63be61 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:54.831344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 480/960] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:54.832788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69bdf8-e815-4d28-b7dc-d5d451b76a11 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:54.947200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69bdf8-e815-4d28-b7dc-d5d451b76a11 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:54.987963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27c442-114d-4806-9b38-657843feef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:54.988627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27c442-114d-4806-9b38-657843feef26 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:54.988902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27c442-114d-4806-9b38-657843feef26 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:54.989202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27c442-114d-4806-9b38-657843feef26 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:54.998265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27c442-114d-4806-9b38-657843feef26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:54.999820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27c442-114d-4806-9b38-657843feef26 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-ca27c442-114d-4806-9b38-657843feef26 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:55.011898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:55.012901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 17:51:55.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:55.017890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 481/961] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:51:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:55.019268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dea3da-8f11-40ec-9282-4e062d63be61 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:55.060912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:55.123181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dea3da-8f11-40ec-9282-4e062d63be61 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:55.158766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bce7f-afdc-4873-af8b-e88c95042455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:55.159436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bce7f-afdc-4873-af8b-e88c95042455 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:55.159775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bce7f-afdc-4873-af8b-e88c95042455 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:55.160041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bce7f-afdc-4873-af8b-e88c95042455 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:55.160839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bce7f-afdc-4873-af8b-e88c95042455 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:55.161142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bce7f-afdc-4873-af8b-e88c95042455 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:55.162559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bce7f-afdc-4873-af8b-e88c95042455 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:55.184793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 481/962] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 17:51:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:55.187012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27c442-114d-4806-9b38-657843feef26 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:55.241598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:55.295335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27c442-114d-4806-9b38-657843feef26 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:55.331624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3f4ed-ebe4-4dc7-a5c6-ff806e3080aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:55.332228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3f4ed-ebe4-4dc7-a5c6-ff806e3080aa None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:55.332520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3f4ed-ebe4-4dc7-a5c6-ff806e3080aa None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:55.332890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3f4ed-ebe4-4dc7-a5c6-ff806e3080aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:55.403973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3f4ed-ebe4-4dc7-a5c6-ff806e3080aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:55.405447 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3f4ed-ebe4-4dc7-a5c6-ff806e3080aa None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:55.445781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d8e35-43f4-456e-beb2-5fdb36c15e01 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:55.446107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d8e35-43f4-456e-beb2-5fdb36c15e01 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:55.446312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d8e35-43f4-456e-beb2-5fdb36c15e01 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:55.447889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 482/963] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:55.452927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a01b6-216e-40b3-aac8-e263dc564254 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:55.453200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a01b6-216e-40b3-aac8-e263dc564254 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:55.453752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a01b6-216e-40b3-aac8-e263dc564254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:55.455434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:55.471674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-468a01b6-216e-40b3-aac8-e263dc564254 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:55.472884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-468a01b6-216e-40b3-aac8-e263dc564254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:55.525423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 482/964] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:51:55 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:51:55.530505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3f4ed-ebe4-4dc7-a5c6-ff806e3080aa None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:55.551689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 483/965] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:55.574959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a01b6-216e-40b3-aac8-e263dc564254 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:55.603041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:55.651932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3f4ed-ebe4-4dc7-a5c6-ff806e3080aa None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:55.700855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fa6f6-433c-4497-bfc0-1759ff18f51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:55.701483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fa6f6-433c-4497-bfc0-1759ff18f51d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:55.701760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fa6f6-433c-4497-bfc0-1759ff18f51d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:55.702028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fa6f6-433c-4497-bfc0-1759ff18f51d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:55.702370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a01b6-216e-40b3-aac8-e263dc564254 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:55.714268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fa6f6-433c-4497-bfc0-1759ff18f51d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:55.716347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fa6f6-433c-4497-bfc0-1759ff18f51d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:55.727223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-0a2fa6f6-433c-4497-bfc0-1759ff18f51d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:51:55.735564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:55.744223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:51:55.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:51:55.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:51:55.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:55.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 483/966] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:55.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c9f03-4a0e-4903-9beb-e989327c8242 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:55.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c9f03-4a0e-4903-9beb-e989327c8242 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:55.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c9f03-4a0e-4903-9beb-e989327c8242 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:55.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 484/967] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:55.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c9f03-4a0e-4903-9beb-e989327c8242 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:55.747347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a4db3-c934-4b37-8211-f4250ded2dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:55.748082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a4db3-c934-4b37-8211-f4250ded2dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:55.748448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a4db3-c934-4b37-8211-f4250ded2dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:55.748835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a4db3-c934-4b37-8211-f4250ded2dd7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:55.749531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a4db3-c934-4b37-8211-f4250ded2dd7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:55.750977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a4db3-c934-4b37-8211-f4250ded2dd7 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-887a4db3-c934-4b37-8211-f4250ded2dd7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:55.760021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:55.764929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:55.765789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 484/968] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:55.767155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe6f74-9342-4d95-b280-a8dad9484557 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:55.767155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe6f74-9342-4d95-b280-a8dad9484557 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:55.767155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe6f74-9342-4d95-b280-a8dad9484557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:55.782111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ecfe6f74-9342-4d95-b280-a8dad9484557 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:55.783211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ecfe6f74-9342-4d95-b280-a8dad9484557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:55.860708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 485/969] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:55.860708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe6f74-9342-4d95-b280-a8dad9484557 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:55.866322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c9f03-4a0e-4903-9beb-e989327c8242 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:55.877329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:55.906144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f678c4-2fe9-4a8b-964d-d33b6b0a16ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:55.906748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f678c4-2fe9-4a8b-964d-d33b6b0a16ad None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:55.907076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f678c4-2fe9-4a8b-964d-d33b6b0a16ad None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:55.907352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f678c4-2fe9-4a8b-964d-d33b6b0a16ad None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:55.907994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f678c4-2fe9-4a8b-964d-d33b6b0a16ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:55.908295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f678c4-2fe9-4a8b-964d-d33b6b0a16ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:55.909758 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f678c4-2fe9-4a8b-964d-d33b6b0a16ad None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:55.926797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 485/970] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:55.926797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f678c4-2fe9-4a8b-964d-d33b6b0a16ad None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:55.974488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe6f74-9342-4d95-b280-a8dad9484557 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:55.990733 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:56.009549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2bbfd-2e6d-4603-9c04-142970339425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:56.010156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2bbfd-2e6d-4603-9c04-142970339425 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.010424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2bbfd-2e6d-4603-9c04-142970339425 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.011208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2bbfd-2e6d-4603-9c04-142970339425 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.011828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2bbfd-2e6d-4603-9c04-142970339425 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:56.012123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2bbfd-2e6d-4603-9c04-142970339425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:56.013538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2bbfd-2e6d-4603-9c04-142970339425 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:56.023191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 486/971] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:51:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:56.023191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2bbfd-2e6d-4603-9c04-142970339425 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:56.037202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f678c4-2fe9-4a8b-964d-d33b6b0a16ad None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:56.073926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:56.074601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.074936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.075265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.076053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:56.077560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:56.102232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:56.122726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2bbfd-2e6d-4603-9c04-142970339425 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:56.139658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.sql.core [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 17:51:56.139658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 17:51:56.139658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [parameters: {'user_id': '4090dd1c29894712857886afa2be227e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 17:51:56.139658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80402) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application conn.query(q) Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result.read() Aug 08 17:51:56.146186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 17:51:56.150950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application next(self.gen) Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application yield resource Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application next(self.gen) Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise self.value Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application session.commit() Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 17:51:56.152610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.session.flush() Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application compat.raise_( Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 17:51:56.153753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 17:51:56.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application conn.query(q) Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application result.read() Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 17:51:56.155946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [parameters: {'user_id': '4090dd1c29894712857886afa2be227e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:56.157050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 17:51:56.158044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 17:51:56.159227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 17:51:56.159227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 17:51:56.159227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 17:51:56.159227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 17:51:56.159227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 17:51:56.159227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:56.159227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 486/972] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 17:51:55 2022] POST /identity/v3/users => generated 160 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:56.159227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:56.169743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b2774-9983-4e25-80dc-79f4a6909b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:56.170372 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b2774-9983-4e25-80dc-79f4a6909b0a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.170701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b2774-9983-4e25-80dc-79f4a6909b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.170965 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b2774-9983-4e25-80dc-79f4a6909b0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.177837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b2774-9983-4e25-80dc-79f4a6909b0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:56.179425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b2774-9983-4e25-80dc-79f4a6909b0a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-a77b2774-9983-4e25-80dc-79f4a6909b0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:51:56.191215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:56.192227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:56.193348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:51:56.193348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:51:56.193348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:51:56.193348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:51:56.193348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:51:56.193348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:51:56.193348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:51:56.193348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:56.195544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 487/973] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:56.196426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b2774-9983-4e25-80dc-79f4a6909b0a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:56.268151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a647e09-641a-4c7a-8422-78e03e9c9b13 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:56.291809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:56.302328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b2774-9983-4e25-80dc-79f4a6909b0a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:56.302873 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a94df-ee93-4ba6-b12e-e2e944d72824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:56.307622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a94df-ee93-4ba6-b12e-e2e944d72824 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.307622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a94df-ee93-4ba6-b12e-e2e944d72824 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.307622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a94df-ee93-4ba6-b12e-e2e944d72824 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.307622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a94df-ee93-4ba6-b12e-e2e944d72824 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:56.307622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a94df-ee93-4ba6-b12e-e2e944d72824 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:56.307622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a94df-ee93-4ba6-b12e-e2e944d72824 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:56.319142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 487/974] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:51:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:56.319142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a94df-ee93-4ba6-b12e-e2e944d72824 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:56.348086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef577cd-f46f-406e-a017-88f09adf74c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:56.349044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef577cd-f46f-406e-a017-88f09adf74c2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.349461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef577cd-f46f-406e-a017-88f09adf74c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.349864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef577cd-f46f-406e-a017-88f09adf74c2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.357930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef577cd-f46f-406e-a017-88f09adf74c2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:56.360039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef577cd-f46f-406e-a017-88f09adf74c2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:56.362396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-bef577cd-f46f-406e-a017-88f09adf74c2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:51:56.374967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:56.376184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 17:51:56.377533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:56.381067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 488/975] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:51:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:56.381667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef577cd-f46f-406e-a017-88f09adf74c2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:56.430593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a94df-ee93-4ba6-b12e-e2e944d72824 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:56.467347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5f012-35bc-431d-a918-25859e338d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:56.467943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5f012-35bc-431d-a918-25859e338d5f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.468215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5f012-35bc-431d-a918-25859e338d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.468480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5f012-35bc-431d-a918-25859e338d5f None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.469182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5f012-35bc-431d-a918-25859e338d5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:56.469478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5f012-35bc-431d-a918-25859e338d5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:56.470893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5f012-35bc-431d-a918-25859e338d5f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:56.484837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef577cd-f46f-406e-a017-88f09adf74c2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:56.488623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 488/976] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:51:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:56.489544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5f012-35bc-431d-a918-25859e338d5f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:56.521549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c68332-9af3-49a0-a144-57bf672ffc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:56.522256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c68332-9af3-49a0-a144-57bf672ffc85 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.522534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c68332-9af3-49a0-a144-57bf672ffc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.522800 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c68332-9af3-49a0-a144-57bf672ffc85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.523440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c68332-9af3-49a0-a144-57bf672ffc85 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:56.524843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c68332-9af3-49a0-a144-57bf672ffc85 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:56.539820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:56.590284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5f012-35bc-431d-a918-25859e338d5f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:56.619803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 489/977] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 17:51:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:56.621096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c68332-9af3-49a0-a144-57bf672ffc85 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:56.639489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8e58b-ca78-4413-8605-e88693d9348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:56.640135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8e58b-ca78-4413-8605-e88693d9348a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.640524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8e58b-ca78-4413-8605-e88693d9348a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.640704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8e58b-ca78-4413-8605-e88693d9348a None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.641404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8e58b-ca78-4413-8605-e88693d9348a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:56.641696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8e58b-ca78-4413-8605-e88693d9348a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:56.643191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8e58b-ca78-4413-8605-e88693d9348a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:56.692532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 489/978] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 17:51:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:56.692532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9394c-1c07-444b-8bd9-af9b772c16ef None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.692532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9394c-1c07-444b-8bd9-af9b772c16ef None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.692532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9394c-1c07-444b-8bd9-af9b772c16ef None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.692532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 490/979] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:56.692532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d7054-ecc0-482f-9016-e3656167f898 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.692532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d7054-ecc0-482f-9016-e3656167f898 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.693205 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d7054-ecc0-482f-9016-e3656167f898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.710512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:56.714346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-5c1d7054-ecc0-482f-9016-e3656167f898 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:56.716399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5c1d7054-ecc0-482f-9016-e3656167f898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:56.781838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c68332-9af3-49a0-a144-57bf672ffc85 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:56.820932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 491/980] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:51:56.838980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077acaba-45e4-448f-890d-8f32801f8eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:56.839601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077acaba-45e4-448f-890d-8f32801f8eed None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:56.839880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077acaba-45e4-448f-890d-8f32801f8eed None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:56.840150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077acaba-45e4-448f-890d-8f32801f8eed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:56.841607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077acaba-45e4-448f-890d-8f32801f8eed None admin] RBAC: Adding query filter params (role.id=b81578bacade480d94c41559bc214637, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=4f0ca85ea7974962b8115428563aef8f) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:56.841607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077acaba-45e4-448f-890d-8f32801f8eed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:56.842343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d7054-ecc0-482f-9016-e3656167f898 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:56.842674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077acaba-45e4-448f-890d-8f32801f8eed None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:56.849584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 490/981] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:51:56 2022] GET /identity/v3/role_assignments?user.id=4f0ca85ea7974962b8115428563aef8f&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=b81578bacade480d94c41559bc214637 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:56.880607 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:56.971831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d7054-ecc0-482f-9016-e3656167f898 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:57.011709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f797e0-2dd4-4e2a-8aa2-0d11ac8434cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:57.012476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f797e0-2dd4-4e2a-8aa2-0d11ac8434cc None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:57.012886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f797e0-2dd4-4e2a-8aa2-0d11ac8434cc None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:57.013255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f797e0-2dd4-4e2a-8aa2-0d11ac8434cc None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:57.014017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f797e0-2dd4-4e2a-8aa2-0d11ac8434cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:57.015496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f797e0-2dd4-4e2a-8aa2-0d11ac8434cc None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:57.117605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 492/982] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:51:56 2022] POST /identity/v3/services => generated 242 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:51:58.711147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a3844-4059-4a10-9fc8-5d76191b8025 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:58.711772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a3844-4059-4a10-9fc8-5d76191b8025 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:58.711772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a3844-4059-4a10-9fc8-5d76191b8025 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:58.713222 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 491/983] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:58.718822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ced866-428a-4f3e-903c-51de80ac06e7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:58.719263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ced866-428a-4f3e-903c-51de80ac06e7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:58.719350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ced866-428a-4f3e-903c-51de80ac06e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:58.745439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b7ced866-428a-4f3e-903c-51de80ac06e7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:58.747012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b7ced866-428a-4f3e-903c-51de80ac06e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:58.836560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 493/984] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:58.858897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a3844-4059-4a10-9fc8-5d76191b8025 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:58.941708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63f49e-78c4-418d-9f88-c95a07211e51 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:58.941990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63f49e-78c4-418d-9f88-c95a07211e51 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:58.942270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63f49e-78c4-418d-9f88-c95a07211e51 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:58.947605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 494/985] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:51:58.949333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c90c7e-f3c5-4cd7-a72c-0b22f787d199 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:58.949610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c90c7e-f3c5-4cd7-a72c-0b22f787d199 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:58.949877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c90c7e-f3c5-4cd7-a72c-0b22f787d199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:58.969566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-91c90c7e-f3c5-4cd7-a72c-0b22f787d199 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:58.970678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-91c90c7e-f3c5-4cd7-a72c-0b22f787d199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:58.972662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a3844-4059-4a10-9fc8-5d76191b8025 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:58.982491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:58.993362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:59.012958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0f49f-47d4-4a58-ae5f-529ef99b4518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.013586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0f49f-47d4-4a58-ae5f-529ef99b4518 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.013861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0f49f-47d4-4a58-ae5f-529ef99b4518 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.014482 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0f49f-47d4-4a58-ae5f-529ef99b4518 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.014707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0f49f-47d4-4a58-ae5f-529ef99b4518 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.016161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0f49f-47d4-4a58-ae5f-529ef99b4518 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-edf0f49f-47d4-4a58-ae5f-529ef99b4518 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:59.024789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:59.025900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:59.027883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 492/986] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:51:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:59.032570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0f49f-47d4-4a58-ae5f-529ef99b4518 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.046983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 495/987] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:59.072709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c90c7e-f3c5-4cd7-a72c-0b22f787d199 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.139285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0f49f-47d4-4a58-ae5f-529ef99b4518 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.174129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e71247-3dea-440e-b3eb-19a690089c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.174722 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e71247-3dea-440e-b3eb-19a690089c9c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.174989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e71247-3dea-440e-b3eb-19a690089c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.175249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e71247-3dea-440e-b3eb-19a690089c9c None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.175868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e71247-3dea-440e-b3eb-19a690089c9c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:59.176082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c90c7e-f3c5-4cd7-a72c-0b22f787d199 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.176181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e71247-3dea-440e-b3eb-19a690089c9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.177537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e71247-3dea-440e-b3eb-19a690089c9c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.182130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 493/988] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:59.183932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59ce0e-f089-41b8-9c83-12ea2b415290 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.184206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59ce0e-f089-41b8-9c83-12ea2b415290 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.184471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59ce0e-f089-41b8-9c83-12ea2b415290 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.187994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 494/989] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:51:59.188651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59ce0e-f089-41b8-9c83-12ea2b415290 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.211047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f248c72-1756-416b-be06-96079c191c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.211671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f248c72-1756-416b-be06-96079c191c72 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.211946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f248c72-1756-416b-be06-96079c191c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.212229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f248c72-1756-416b-be06-96079c191c72 None admin] PATH_INFO: `/v3/services/image` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.212839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f248c72-1756-416b-be06-96079c191c72 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.214229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f248c72-1756-416b-be06-96079c191c72 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9f248c72-1756-416b-be06-96079c191c72 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:59.223097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:59.224334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:51:59.225244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 496/990] 10.222.0.50 () {56 vars in 1105 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:51:59.226165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c773-9225-41bd-a90f-06d817b8de34 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.226165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c773-9225-41bd-a90f-06d817b8de34 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.226165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c773-9225-41bd-a90f-06d817b8de34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.238609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:59.243282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e651c773-9225-41bd-a90f-06d817b8de34 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:51:59.244460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e651c773-9225-41bd-a90f-06d817b8de34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:51:59.250489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:59.291240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59ce0e-f089-41b8-9c83-12ea2b415290 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.311145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 497/991] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:51:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:51:59.312163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651c773-9225-41bd-a90f-06d817b8de34 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.326578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdd87e-01e7-426c-9800-66e55c8f2cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.327173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdd87e-01e7-426c-9800-66e55c8f2cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.327445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdd87e-01e7-426c-9800-66e55c8f2cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.327706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdd87e-01e7-426c-9800-66e55c8f2cd9 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.328605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdd87e-01e7-426c-9800-66e55c8f2cd9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:59.328901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdd87e-01e7-426c-9800-66e55c8f2cd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.330258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdd87e-01e7-426c-9800-66e55c8f2cd9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.335362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 495/992] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:59.336571 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdd87e-01e7-426c-9800-66e55c8f2cd9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.412229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651c773-9225-41bd-a90f-06d817b8de34 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.431742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdd87e-01e7-426c-9800-66e55c8f2cd9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.442262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:59.446422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad728d-69d6-44b3-bd92-f4e5fd52fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.447018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad728d-69d6-44b3-bd92-f4e5fd52fa84 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.447288 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad728d-69d6-44b3-bd92-f4e5fd52fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.447552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad728d-69d6-44b3-bd92-f4e5fd52fa84 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.448200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad728d-69d6-44b3-bd92-f4e5fd52fa84 None admin] RBAC: Adding query filter params (name=image) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:59.448492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad728d-69d6-44b3-bd92-f4e5fd52fa84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.449894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad728d-69d6-44b3-bd92-f4e5fd52fa84 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.455054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 498/993] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:59.461354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad728d-69d6-44b3-bd92-f4e5fd52fa84 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.461354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:59.466224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55827a-4169-43cd-8e53-cc4890414489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.466845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55827a-4169-43cd-8e53-cc4890414489 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.467118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55827a-4169-43cd-8e53-cc4890414489 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.467380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55827a-4169-43cd-8e53-cc4890414489 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.467954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55827a-4169-43cd-8e53-cc4890414489 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.469331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55827a-4169-43cd-8e53-cc4890414489 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-5d55827a-4169-43cd-8e53-cc4890414489 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:51:59.477795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:51:59.478781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:51:59.480699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 496/994] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:51:59.481710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55827a-4169-43cd-8e53-cc4890414489 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.559057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad728d-69d6-44b3-bd92-f4e5fd52fa84 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.578162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55827a-4169-43cd-8e53-cc4890414489 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.593291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622d85c-76bc-4033-91f1-8145c9594843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.593888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622d85c-76bc-4033-91f1-8145c9594843 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.594162 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622d85c-76bc-4033-91f1-8145c9594843 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.594423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622d85c-76bc-4033-91f1-8145c9594843 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.595119 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622d85c-76bc-4033-91f1-8145c9594843 None admin] RBAC: Adding query filter params (interface=admin, service_id=4740c5a94c6341b38753f0a266315ae1, region_id=RegionOne) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:59.595421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622d85c-76bc-4033-91f1-8145c9594843 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.596802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622d85c-76bc-4033-91f1-8145c9594843 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.601662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 499/995] 10.222.0.50 () {56 vars in 1249 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/endpoints?service_id=4740c5a94c6341b38753f0a266315ae1&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:59.602430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622d85c-76bc-4033-91f1-8145c9594843 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.613213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22e017-6fd9-402a-8223-be495a051fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.613812 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22e017-6fd9-402a-8223-be495a051fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.614086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22e017-6fd9-402a-8223-be495a051fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.614354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22e017-6fd9-402a-8223-be495a051fa5 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.614978 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22e017-6fd9-402a-8223-be495a051fa5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:59.615273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22e017-6fd9-402a-8223-be495a051fa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.616760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22e017-6fd9-402a-8223-be495a051fa5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.621211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 497/996] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:59.621988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22e017-6fd9-402a-8223-be495a051fa5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.676072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:59.684937 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:59.715705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622d85c-76bc-4033-91f1-8145c9594843 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.753426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22e017-6fd9-402a-8223-be495a051fa5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.756047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a91656-5b5d-4bf4-b427-553cd9a7eedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.756765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a91656-5b5d-4bf4-b427-553cd9a7eedd None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.757052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a91656-5b5d-4bf4-b427-553cd9a7eedd None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.757318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a91656-5b5d-4bf4-b427-553cd9a7eedd None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.757958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a91656-5b5d-4bf4-b427-553cd9a7eedd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:59.758258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a91656-5b5d-4bf4-b427-553cd9a7eedd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.759652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a91656-5b5d-4bf4-b427-553cd9a7eedd None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.765949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 500/997] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:51:59.772427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a91656-5b5d-4bf4-b427-553cd9a7eedd None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:51:59.800757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb68f29-1add-4d48-bbf2-76fb2281d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.801348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb68f29-1add-4d48-bbf2-76fb2281d73e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.801627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb68f29-1add-4d48-bbf2-76fb2281d73e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.801893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb68f29-1add-4d48-bbf2-76fb2281d73e None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.802504 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb68f29-1add-4d48-bbf2-76fb2281d73e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.804013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb68f29-1add-4d48-bbf2-76fb2281d73e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.810806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 498/998] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/services => generated 1727 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:51:59.894453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a91656-5b5d-4bf4-b427-553cd9a7eedd None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:51:59.927323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:51:59.933314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077f3a-c4d5-4d3a-96d4-7280dd44b759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:51:59.933935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077f3a-c4d5-4d3a-96d4-7280dd44b759 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:51:59.934209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077f3a-c4d5-4d3a-96d4-7280dd44b759 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:51:59.934543 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077f3a-c4d5-4d3a-96d4-7280dd44b759 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:51:59.935185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077f3a-c4d5-4d3a-96d4-7280dd44b759 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:51:59.935566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077f3a-c4d5-4d3a-96d4-7280dd44b759 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:51:59.936925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077f3a-c4d5-4d3a-96d4-7280dd44b759 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:51:59.947346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 501/999] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:51:59.949626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb68f29-1add-4d48-bbf2-76fb2281d73e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:00.054339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb68f29-1add-4d48-bbf2-76fb2281d73e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:00.055902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:00.090320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455711-66a5-4e22-88fb-1fa02d8ac4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:00.092431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455711-66a5-4e22-88fb-1fa02d8ac4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:00.092431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455711-66a5-4e22-88fb-1fa02d8ac4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:00.092431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455711-66a5-4e22-88fb-1fa02d8ac4e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:00.092431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455711-66a5-4e22-88fb-1fa02d8ac4e5 None admin] RBAC: Adding query filter params (interface=public, service_id=1f2683cf704d45ea9d258bdd7716f57c, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:00.092431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455711-66a5-4e22-88fb-1fa02d8ac4e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:00.093786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455711-66a5-4e22-88fb-1fa02d8ac4e5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:00.099153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 499/1000] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 17:51:59 2022] GET /identity/v3/endpoints?service_id=1f2683cf704d45ea9d258bdd7716f57c&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:00.104699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077f3a-c4d5-4d3a-96d4-7280dd44b759 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:00.206309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077f3a-c4d5-4d3a-96d4-7280dd44b759 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:00.239329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3573a1-a316-4f5e-b450-ca4a6d475dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:00.239935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3573a1-a316-4f5e-b450-ca4a6d475dce None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:00.240205 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3573a1-a316-4f5e-b450-ca4a6d475dce None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:00.240467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3573a1-a316-4f5e-b450-ca4a6d475dce None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:00.241094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3573a1-a316-4f5e-b450-ca4a6d475dce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:00.242443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3573a1-a316-4f5e-b450-ca4a6d475dce None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:00.256580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 502/1001] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 17:52:00 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:52:01.647208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb2083-b5b0-4b6a-af3d-985b4b9a2872 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:01.647749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb2083-b5b0-4b6a-af3d-985b4b9a2872 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:01.647749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb2083-b5b0-4b6a-af3d-985b4b9a2872 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:01.649140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 500/1002] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:01.654224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020da08e-b0d2-444a-81c0-2b0ff9a1c658 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:01.654668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020da08e-b0d2-444a-81c0-2b0ff9a1c658 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:01.654750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020da08e-b0d2-444a-81c0-2b0ff9a1c658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:01.672682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-020da08e-b0d2-444a-81c0-2b0ff9a1c658 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:01.673863 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-020da08e-b0d2-444a-81c0-2b0ff9a1c658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:01.731250 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:01.742261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 503/1003] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:01.766763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb2083-b5b0-4b6a-af3d-985b4b9a2872 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:01.840795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fb559-ae81-4074-9cc3-a0899f3927fd None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:01.841140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fb559-ae81-4074-9cc3-a0899f3927fd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:01.841345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fb559-ae81-4074-9cc3-a0899f3927fd None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:01.842899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 504/1004] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:52:01.847636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f867b5-40a2-4be8-92a1-e663c97b7bb6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:01.847917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f867b5-40a2-4be8-92a1-e663c97b7bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:01.848188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f867b5-40a2-4be8-92a1-e663c97b7bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:01.866042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-54f867b5-40a2-4be8-92a1-e663c97b7bb6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:01.867275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-54f867b5-40a2-4be8-92a1-e663c97b7bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:01.882749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb2083-b5b0-4b6a-af3d-985b4b9a2872 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:01.919919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d5349-b7fb-4192-ad10-60e5354bfd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:01.920440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d5349-b7fb-4192-ad10-60e5354bfd6d None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:01.920764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d5349-b7fb-4192-ad10-60e5354bfd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:01.921039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d5349-b7fb-4192-ad10-60e5354bfd6d None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:01.921675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d5349-b7fb-4192-ad10-60e5354bfd6d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:01.923064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d5349-b7fb-4192-ad10-60e5354bfd6d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:01.940618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 505/1005] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:52:01.964354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f867b5-40a2-4be8-92a1-e663c97b7bb6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:01.987232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 501/1006] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:52:01 2022] POST /identity/v3/services => generated 242 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:52:02.081023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f867b5-40a2-4be8-92a1-e663c97b7bb6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:02.102754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:02.125564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef98741-5518-4f5f-a1bc-7a3b1796e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:02.126201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef98741-5518-4f5f-a1bc-7a3b1796e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:02.126478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef98741-5518-4f5f-a1bc-7a3b1796e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:02.126742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef98741-5518-4f5f-a1bc-7a3b1796e9a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:02.127320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef98741-5518-4f5f-a1bc-7a3b1796e9a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:02.128735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef98741-5518-4f5f-a1bc-7a3b1796e9a3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4ef98741-5518-4f5f-a1bc-7a3b1796e9a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:02.138752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:02.139952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 17:52:02.141045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:02.143835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 506/1007] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 17:52:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:52:02.149383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d5349-b7fb-4192-ad10-60e5354bfd6d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:02.180189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:02.252658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5eee1-a9ec-4a0f-9e96-588ab95c4d65 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:02.252943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5eee1-a9ec-4a0f-9e96-588ab95c4d65 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:02.253218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5eee1-a9ec-4a0f-9e96-588ab95c4d65 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:02.254555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 507/1008] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:52:02.257509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d5349-b7fb-4192-ad10-60e5354bfd6d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:02.259400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2253ba-366b-412b-aa2d-95d65ebab03f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:02.259678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2253ba-366b-412b-aa2d-95d65ebab03f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:02.259943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2253ba-366b-412b-aa2d-95d65ebab03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:02.277516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-cc2253ba-366b-412b-aa2d-95d65ebab03f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:02.278695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cc2253ba-366b-412b-aa2d-95d65ebab03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:02.293820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9f981-c8cd-436d-b9f2-83bcde530205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:02.294620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9f981-c8cd-436d-b9f2-83bcde530205 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:02.295078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9f981-c8cd-436d-b9f2-83bcde530205 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:02.295619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9f981-c8cd-436d-b9f2-83bcde530205 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:02.296527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9f981-c8cd-436d-b9f2-83bcde530205 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:02.296989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9f981-c8cd-436d-b9f2-83bcde530205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:02.298481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9f981-c8cd-436d-b9f2-83bcde530205 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:02.304095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 502/1009] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:52:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:02.310192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9f981-c8cd-436d-b9f2-83bcde530205 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:02.350138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 508/1010] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:02 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:52:02.372914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2253ba-366b-412b-aa2d-95d65ebab03f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:02.406172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:02.416277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9f981-c8cd-436d-b9f2-83bcde530205 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:02.451263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d37b3b-85df-4fcf-9446-99acef5da3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:02.451790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d37b3b-85df-4fcf-9446-99acef5da3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:02.452064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d37b3b-85df-4fcf-9446-99acef5da3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:02.452329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d37b3b-85df-4fcf-9446-99acef5da3d5 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:02.452987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d37b3b-85df-4fcf-9446-99acef5da3d5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:02.453532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d37b3b-85df-4fcf-9446-99acef5da3d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:02.454881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d37b3b-85df-4fcf-9446-99acef5da3d5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:02.464585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 503/1011] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 17:52:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:02.466347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d37b3b-85df-4fcf-9446-99acef5da3d5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:02.478256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2253ba-366b-412b-aa2d-95d65ebab03f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:02.493589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:02.520337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48c03-b0c7-45b7-94c0-93c980db4bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:02.521095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48c03-b0c7-45b7-94c0-93c980db4bcb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:02.521492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48c03-b0c7-45b7-94c0-93c980db4bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:02.521888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48c03-b0c7-45b7-94c0-93c980db4bcb None admin] PATH_INFO: `/v3/services/compute` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:02.522584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48c03-b0c7-45b7-94c0-93c980db4bcb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:02.523941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48c03-b0c7-45b7-94c0-93c980db4bcb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5cc48c03-b0c7-45b7-94c0-93c980db4bcb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:02.534520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:02.535734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:02.536979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 509/1012] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:52:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:52:02.541457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48c03-b0c7-45b7-94c0-93c980db4bcb None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:02.585755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d37b3b-85df-4fcf-9446-99acef5da3d5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:02.620810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d41be-81c5-4f4d-9c01-88505e826185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:02.621414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d41be-81c5-4f4d-9c01-88505e826185 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:02.621690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d41be-81c5-4f4d-9c01-88505e826185 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:02.621971 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d41be-81c5-4f4d-9c01-88505e826185 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:02.622762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d41be-81c5-4f4d-9c01-88505e826185 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:02.624343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d41be-81c5-4f4d-9c01-88505e826185 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:02.644789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48c03-b0c7-45b7-94c0-93c980db4bcb None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:02.678194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee11b1-8808-4927-a2e4-e3e8c8e2f407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:02.678838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee11b1-8808-4927-a2e4-e3e8c8e2f407 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:02.679037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee11b1-8808-4927-a2e4-e3e8c8e2f407 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:02.679299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee11b1-8808-4927-a2e4-e3e8c8e2f407 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:02.679911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee11b1-8808-4927-a2e4-e3e8c8e2f407 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:02.680204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee11b1-8808-4927-a2e4-e3e8c8e2f407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:02.681591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee11b1-8808-4927-a2e4-e3e8c8e2f407 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:02.686315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 510/1013] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:52:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:02.691817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee11b1-8808-4927-a2e4-e3e8c8e2f407 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:02.707067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 504/1014] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 17:52:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:52:02.735089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:02.812663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee11b1-8808-4927-a2e4-e3e8c8e2f407 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:02.858266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cac756-3cb6-4cb0-bf75-66d8d1b50892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:02.858881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cac756-3cb6-4cb0-bf75-66d8d1b50892 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:02.859156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cac756-3cb6-4cb0-bf75-66d8d1b50892 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:02.859424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cac756-3cb6-4cb0-bf75-66d8d1b50892 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:02.860049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cac756-3cb6-4cb0-bf75-66d8d1b50892 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:02.860348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cac756-3cb6-4cb0-bf75-66d8d1b50892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:02.861810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cac756-3cb6-4cb0-bf75-66d8d1b50892 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:02.867413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 511/1015] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 17:52:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:02.872915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d41be-81c5-4f4d-9c01-88505e826185 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:02.890862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:02.979379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d41be-81c5-4f4d-9c01-88505e826185 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:03.013697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b861d2-1561-4dc6-b321-23fe2d0ff3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:03.014555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b861d2-1561-4dc6-b321-23fe2d0ff3f6 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:03.015032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b861d2-1561-4dc6-b321-23fe2d0ff3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:03.015447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b861d2-1561-4dc6-b321-23fe2d0ff3f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:03.016255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b861d2-1561-4dc6-b321-23fe2d0ff3f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:03.017783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b861d2-1561-4dc6-b321-23fe2d0ff3f6 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:03.146570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 505/1016] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 17:52:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:52:04.037506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcefd3-cb7a-4ab0-aa1b-f516f266a3b9 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:04.038038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcefd3-cb7a-4ab0-aa1b-f516f266a3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:04.038038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcefd3-cb7a-4ab0-aa1b-f516f266a3b9 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:04.039522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 512/1017] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:04.045030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6564-65a5-402b-885b-c08f3c5c46b0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:04.045428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6564-65a5-402b-885b-c08f3c5c46b0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:04.045588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6564-65a5-402b-885b-c08f3c5c46b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:04.064282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e29c6564-65a5-402b-885b-c08f3c5c46b0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:04.065417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e29c6564-65a5-402b-885b-c08f3c5c46b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:04.128192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:04.138529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 506/1018] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:04.161503 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcefd3-cb7a-4ab0-aa1b-f516f266a3b9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:04.262679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:04.267952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcefd3-cb7a-4ab0-aa1b-f516f266a3b9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:04.302991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047609e-b2ed-498d-95a6-66514098e177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:04.303613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047609e-b2ed-498d-95a6-66514098e177 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:04.303885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047609e-b2ed-498d-95a6-66514098e177 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:04.304148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047609e-b2ed-498d-95a6-66514098e177 None admin] PATH_INFO: `/v3/services/image` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:04.304752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047609e-b2ed-498d-95a6-66514098e177 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:04.306112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047609e-b2ed-498d-95a6-66514098e177 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-a047609e-b2ed-498d-95a6-66514098e177 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:04.314987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:04.315998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 17:52:04.317272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:04.320665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 513/1019] 10.222.0.50 () {56 vars in 1105 bytes} [Mon Aug 8 17:52:04 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:04.322283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6564-65a5-402b-885b-c08f3c5c46b0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:04.423916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6564-65a5-402b-885b-c08f3c5c46b0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:04.458501 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21264dea-5b7d-402f-aba3-30812c309820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:04.459092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21264dea-5b7d-402f-aba3-30812c309820 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:04.459368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21264dea-5b7d-402f-aba3-30812c309820 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:04.459629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21264dea-5b7d-402f-aba3-30812c309820 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:04.460242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21264dea-5b7d-402f-aba3-30812c309820 None admin] RBAC: Adding query filter params (name=image) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:04.460554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21264dea-5b7d-402f-aba3-30812c309820 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:04.462188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21264dea-5b7d-402f-aba3-30812c309820 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:04.467714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 507/1020] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:04.473301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047609e-b2ed-498d-95a6-66514098e177 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:04.570534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047609e-b2ed-498d-95a6-66514098e177 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:04.606609 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc20a8-e170-4a07-88bf-0e048226b5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:04.607209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc20a8-e170-4a07-88bf-0e048226b5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:04.607485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc20a8-e170-4a07-88bf-0e048226b5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:04.607752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc20a8-e170-4a07-88bf-0e048226b5d9 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:04.608375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc20a8-e170-4a07-88bf-0e048226b5d9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:04.608702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc20a8-e170-4a07-88bf-0e048226b5d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:04.610063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc20a8-e170-4a07-88bf-0e048226b5d9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:04.620031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 514/1021] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:04.621726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21264dea-5b7d-402f-aba3-30812c309820 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:04.681320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:04.722792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21264dea-5b7d-402f-aba3-30812c309820 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:04.753185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff981d01-ea24-4c56-a0af-d5df5a86e9f8 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:04.753582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff981d01-ea24-4c56-a0af-d5df5a86e9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:04.753755 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff981d01-ea24-4c56-a0af-d5df5a86e9f8 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:04.755180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 515/1022] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:52:04.757047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670311c9-c54f-441c-bfde-bc3b82c02b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:04.757648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670311c9-c54f-441c-bfde-bc3b82c02b58 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:04.757922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670311c9-c54f-441c-bfde-bc3b82c02b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:04.758187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670311c9-c54f-441c-bfde-bc3b82c02b58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:04.758959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670311c9-c54f-441c-bfde-bc3b82c02b58 None admin] RBAC: Adding query filter params (interface=public, service_id=1502ec27966d4b5dbcec98c61c6bd362, region_id=RegionOne) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:04.759268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670311c9-c54f-441c-bfde-bc3b82c02b58 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:04.759785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a634b-5e8b-417b-ba89-d077dca64e90 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:04.760087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a634b-5e8b-417b-ba89-d077dca64e90 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:04.760989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a634b-5e8b-417b-ba89-d077dca64e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:04.761209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670311c9-c54f-441c-bfde-bc3b82c02b58 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:04.771080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 508/1023] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 17:52:04 2022] GET /identity/v3/endpoints?service_id=1502ec27966d4b5dbcec98c61c6bd362&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:04.771848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670311c9-c54f-441c-bfde-bc3b82c02b58 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:04.783619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3e4a634b-5e8b-417b-ba89-d077dca64e90 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:04.784850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3e4a634b-5e8b-417b-ba89-d077dca64e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:04.808967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:04.856826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 516/1024] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:52:04.879829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a634b-5e8b-417b-ba89-d077dca64e90 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:04.890471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670311c9-c54f-441c-bfde-bc3b82c02b58 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:04.914539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:04.926215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab7240-c993-48a2-abf0-1e4c3877faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:04.926823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab7240-c993-48a2-abf0-1e4c3877faf0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:04.927096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab7240-c993-48a2-abf0-1e4c3877faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:04.927358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab7240-c993-48a2-abf0-1e4c3877faf0 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:04.927965 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab7240-c993-48a2-abf0-1e4c3877faf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:04.929348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab7240-c993-48a2-abf0-1e4c3877faf0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:04.936746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 509/1025] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 17:52:04 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:52:04.995895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a634b-5e8b-417b-ba89-d077dca64e90 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:05.032222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52ba82-c43b-4e39-ad14-27765503b527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:05.032888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52ba82-c43b-4e39-ad14-27765503b527 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:05.033165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52ba82-c43b-4e39-ad14-27765503b527 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:05.033431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52ba82-c43b-4e39-ad14-27765503b527 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:05.034584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52ba82-c43b-4e39-ad14-27765503b527 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:05.036037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52ba82-c43b-4e39-ad14-27765503b527 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:05.037873 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 517/1026] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:05.044363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab7240-c993-48a2-abf0-1e4c3877faf0 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:05.148817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab7240-c993-48a2-abf0-1e4c3877faf0 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:05.184089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e69db-75b6-4ace-9f64-a734d9eb3ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:05.185112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e69db-75b6-4ace-9f64-a734d9eb3ea1 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:05.185590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e69db-75b6-4ace-9f64-a734d9eb3ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:05.186072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e69db-75b6-4ace-9f64-a734d9eb3ea1 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:05.186983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e69db-75b6-4ace-9f64-a734d9eb3ea1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:05.188632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e69db-75b6-4ace-9f64-a734d9eb3ea1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:05.272538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 510/1027] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:52:05 2022] POST /identity/v3/projects => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:52:05.282578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a97b39-0b7c-4477-bb8a-a8d6045e2299 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:05.282851 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a97b39-0b7c-4477-bb8a-a8d6045e2299 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:05.283121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a97b39-0b7c-4477-bb8a-a8d6045e2299 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:05.284730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 518/1028] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:05.290836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25b9f1-b2ed-4f83-969a-73d3695cee85 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:05.291176 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25b9f1-b2ed-4f83-969a-73d3695cee85 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:05.291912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25b9f1-b2ed-4f83-969a-73d3695cee85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:05.312318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1c25b9f1-b2ed-4f83-969a-73d3695cee85 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:05.313584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1c25b9f1-b2ed-4f83-969a-73d3695cee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:05.351959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:05.393040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 511/1029] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:05.412169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a97b39-0b7c-4477-bb8a-a8d6045e2299 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:05.438978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:05.519416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a97b39-0b7c-4477-bb8a-a8d6045e2299 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:05.554053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c516b-fbbc-4276-8616-0f0dd4f35bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:05.554678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c516b-fbbc-4276-8616-0f0dd4f35bed None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:05.554961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c516b-fbbc-4276-8616-0f0dd4f35bed None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:05.555229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c516b-fbbc-4276-8616-0f0dd4f35bed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:05.565457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c516b-fbbc-4276-8616-0f0dd4f35bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-eb0c516b-fbbc-4276-8616-0f0dd4f35bed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:05.570180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:05.571139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:05.572054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 519/1030] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 17:52:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:05.578016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25b9f1-b2ed-4f83-969a-73d3695cee85 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:05.680066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25b9f1-b2ed-4f83-969a-73d3695cee85 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:05.716032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039f4c0-586e-4c50-ad86-046d28c2565e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:05.716661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039f4c0-586e-4c50-ad86-046d28c2565e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:05.716935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039f4c0-586e-4c50-ad86-046d28c2565e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:05.717198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039f4c0-586e-4c50-ad86-046d28c2565e None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:05.717856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039f4c0-586e-4c50-ad86-046d28c2565e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:05.718150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039f4c0-586e-4c50-ad86-046d28c2565e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:05.719509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039f4c0-586e-4c50-ad86-046d28c2565e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:05.727578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 512/1031] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 17:52:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:05.733112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c516b-fbbc-4276-8616-0f0dd4f35bed None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:05.827108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:05.843714 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c516b-fbbc-4276-8616-0f0dd4f35bed None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:05.877563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d432-b1f9-493a-815a-d7faa1f66dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:05.878180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d432-b1f9-493a-815a-d7faa1f66dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:05.878454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d432-b1f9-493a-815a-d7faa1f66dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:05.878718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d432-b1f9-493a-815a-d7faa1f66dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:05.889085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12d432-b1f9-493a-815a-d7faa1f66dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:05.890525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12d432-b1f9-493a-815a-d7faa1f66dc4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5f12d432-b1f9-493a-815a-d7faa1f66dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:52:05.905201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:05.906166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:52:05.907144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:52:05.907144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:05.907144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:05.908826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 520/1032] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:05.911880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039f4c0-586e-4c50-ad86-046d28c2565e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:05.943359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:06.012408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039f4c0-586e-4c50-ad86-046d28c2565e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:06.046266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07f730-13b9-4a43-a829-d171e2f3a7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:06.046884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f730-13b9-4a43-a829-d171e2f3a7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:06.047161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f730-13b9-4a43-a829-d171e2f3a7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:06.047430 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f730-13b9-4a43-a829-d171e2f3a7b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:06.048092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07f730-13b9-4a43-a829-d171e2f3a7b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:06.048392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07f730-13b9-4a43-a829-d171e2f3a7b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:06.049833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07f730-13b9-4a43-a829-d171e2f3a7b9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:06.067455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 513/1033] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:06.068725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d432-b1f9-493a-815a-d7faa1f66dc4 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:06.171740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d432-b1f9-493a-815a-d7faa1f66dc4 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:06.205779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b61d7-4f66-4271-8dcd-32824939d664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:06.206336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b61d7-4f66-4271-8dcd-32824939d664 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:06.206610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b61d7-4f66-4271-8dcd-32824939d664 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:06.206952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b61d7-4f66-4271-8dcd-32824939d664 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:06.209662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:06.215410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b61d7-4f66-4271-8dcd-32824939d664 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:06.216875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b61d7-4f66-4271-8dcd-32824939d664 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f23b61d7-4f66-4271-8dcd-32824939d664 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:52:06.227577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:06.233401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:06.234434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 521/1034] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 17:52:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:06.236255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07f730-13b9-4a43-a829-d171e2f3a7b9 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:06.330194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:06.338842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07f730-13b9-4a43-a829-d171e2f3a7b9 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:06.372677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621c78d-b3ef-48d4-b926-85dc3b43e1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:06.373284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621c78d-b3ef-48d4-b926-85dc3b43e1cf None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:06.373559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621c78d-b3ef-48d4-b926-85dc3b43e1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:06.373828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621c78d-b3ef-48d4-b926-85dc3b43e1cf None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:06.374551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621c78d-b3ef-48d4-b926-85dc3b43e1cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:06.374846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621c78d-b3ef-48d4-b926-85dc3b43e1cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:06.376265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621c78d-b3ef-48d4-b926-85dc3b43e1cf None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:06.398630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 514/1035] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 17:52:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:06.399761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b61d7-4f66-4271-8dcd-32824939d664 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:06.503058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b61d7-4f66-4271-8dcd-32824939d664 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:06.535915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a501b5-1450-45f8-82fa-bb1606d0af34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:06.536524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a501b5-1450-45f8-82fa-bb1606d0af34 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:06.536827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a501b5-1450-45f8-82fa-bb1606d0af34 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:06.537093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a501b5-1450-45f8-82fa-bb1606d0af34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:06.600545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:06.608051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a501b5-1450-45f8-82fa-bb1606d0af34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:06.609596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a501b5-1450-45f8-82fa-bb1606d0af34 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:06.714461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 522/1036] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:06 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:06.715908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621c78d-b3ef-48d4-b926-85dc3b43e1cf None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:06.827257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621c78d-b3ef-48d4-b926-85dc3b43e1cf None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:06.862760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39741da-c588-41dd-a2a0-31f9ba7ceba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:06.863371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39741da-c588-41dd-a2a0-31f9ba7ceba9 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:06.863638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39741da-c588-41dd-a2a0-31f9ba7ceba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:06.863897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39741da-c588-41dd-a2a0-31f9ba7ceba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:06.874069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39741da-c588-41dd-a2a0-31f9ba7ceba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:06.875502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39741da-c588-41dd-a2a0-31f9ba7ceba9 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c39741da-c588-41dd-a2a0-31f9ba7ceba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:52:06.889809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:06.890813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:52:06.896436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:52:06.896436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:06.896436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:06.896436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 515/1037] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:06.898060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a501b5-1450-45f8-82fa-bb1606d0af34 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:06.959835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773fc2-2cd2-4228-9686-06bb0456a10f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:06.960117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773fc2-2cd2-4228-9686-06bb0456a10f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:06.960392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773fc2-2cd2-4228-9686-06bb0456a10f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:06.961872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 516/1038] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:52:06.967100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68efaf-776e-479e-b937-ec6b6b601b5a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:06.967378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68efaf-776e-479e-b937-ec6b6b601b5a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:06.967646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68efaf-776e-479e-b937-ec6b6b601b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:06.974036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:06.986956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-af68efaf-776e-479e-b937-ec6b6b601b5a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:06.988134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-af68efaf-776e-479e-b937-ec6b6b601b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:07.001975 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:07.011124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a501b5-1450-45f8-82fa-bb1606d0af34 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.050829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47b606-18c6-47d2-9003-4e5201937512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.051424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47b606-18c6-47d2-9003-4e5201937512 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.051696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47b606-18c6-47d2-9003-4e5201937512 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.051961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47b606-18c6-47d2-9003-4e5201937512 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.052693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47b606-18c6-47d2-9003-4e5201937512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:07.052937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47b606-18c6-47d2-9003-4e5201937512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.054518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47b606-18c6-47d2-9003-4e5201937512 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.059179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 517/1039] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:52:07.068810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 523/1040] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:07.071419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68efaf-776e-479e-b937-ec6b6b601b5a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:07.081912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47b606-18c6-47d2-9003-4e5201937512 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:07.182150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68efaf-776e-479e-b937-ec6b6b601b5a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.186338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47b606-18c6-47d2-9003-4e5201937512 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.214113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:07.219814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54a3b9-f36b-49ff-b363-99cda43ba6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.220564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54a3b9-f36b-49ff-b363-99cda43ba6ee None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.221045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54a3b9-f36b-49ff-b363-99cda43ba6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.221162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54a3b9-f36b-49ff-b363-99cda43ba6ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.222277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39cd69-2912-4b17-a228-43a862d284b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.222923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39cd69-2912-4b17-a228-43a862d284b8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.223197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39cd69-2912-4b17-a228-43a862d284b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.223462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39cd69-2912-4b17-a228-43a862d284b8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.224038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39cd69-2912-4b17-a228-43a862d284b8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.225421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39cd69-2912-4b17-a228-43a862d284b8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.228164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54a3b9-f36b-49ff-b363-99cda43ba6ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.229725 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54a3b9-f36b-49ff-b363-99cda43ba6ee None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-fe39cd69-2912-4b17-a228-43a862d284b8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:07.234342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:07.235556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 17:52:07.236439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:07.240668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 524/1041] 10.222.0.50 () {56 vars in 1105 bytes} [Mon Aug 8 17:52:07 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:52:07.241493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39cd69-2912-4b17-a228-43a862d284b8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ea54a3b9-f36b-49ff-b363-99cda43ba6ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:52:07.243379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:07.244795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:07.245702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:52:07.245702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:52:07.245702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:52:07.245702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:52:07.245702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:52:07.245702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:52:07.245702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:52:07.245702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:07.252630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 518/1042] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:52:07.253321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54a3b9-f36b-49ff-b363-99cda43ba6ee None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:07.265477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:07.347962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39cd69-2912-4b17-a228-43a862d284b8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.355203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54a3b9-f36b-49ff-b363-99cda43ba6ee None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.379912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48460-c1c0-48f4-9107-7c31625c078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.380517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48460-c1c0-48f4-9107-7c31625c078e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.380798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48460-c1c0-48f4-9107-7c31625c078e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.381060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48460-c1c0-48f4-9107-7c31625c078e None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.381669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48460-c1c0-48f4-9107-7c31625c078e None admin] RBAC: Adding query filter params (name=image) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:07.381961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48460-c1c0-48f4-9107-7c31625c078e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.383307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48460-c1c0-48f4-9107-7c31625c078e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.388487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 525/1043] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:07.388624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06767-4558-411b-aa29-8e4c7431f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.389131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06767-4558-411b-aa29-8e4c7431f064 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.389406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06767-4558-411b-aa29-8e4c7431f064 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.389623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bcac2-5ee0-4a48-80c7-94846bbe0967 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.389897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bcac2-5ee0-4a48-80c7-94846bbe0967 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.390060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06767-4558-411b-aa29-8e4c7431f064 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.390153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bcac2-5ee0-4a48-80c7-94846bbe0967 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.390720 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06767-4558-411b-aa29-8e4c7431f064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:07.391019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06767-4558-411b-aa29-8e4c7431f064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.392538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 526/1044] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:07.392659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06767-4558-411b-aa29-8e4c7431f064 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.393879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bcac2-5ee0-4a48-80c7-94846bbe0967 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:07.405829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 519/1045] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:52:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:07.407094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0548c-1194-4711-a126-38b6e92ce994 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.407374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0548c-1194-4711-a126-38b6e92ce994 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.407644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0548c-1194-4711-a126-38b6e92ce994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.424893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d9c0548c-1194-4711-a126-38b6e92ce994 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:07.426073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d9c0548c-1194-4711-a126-38b6e92ce994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:07.473004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:07.478827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:07.492079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 520/1046] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:07.492882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0548c-1194-4711-a126-38b6e92ce994 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:07.494915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bcac2-5ee0-4a48-80c7-94846bbe0967 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.530463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae5748-d38a-4712-9573-963655668835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.531056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae5748-d38a-4712-9573-963655668835 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.531325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae5748-d38a-4712-9573-963655668835 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.531584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae5748-d38a-4712-9573-963655668835 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.532214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae5748-d38a-4712-9573-963655668835 None admin] RBAC: Adding query filter params (type=image) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:07.532525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae5748-d38a-4712-9573-963655668835 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.534784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae5748-d38a-4712-9573-963655668835 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.539355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 527/1047] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:07.540107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae5748-d38a-4712-9573-963655668835 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:07.601306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0548c-1194-4711-a126-38b6e92ce994 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.635887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78311232-640e-4f51-adc2-24934c2f6eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.636235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae5748-d38a-4712-9573-963655668835 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.636475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78311232-640e-4f51-adc2-24934c2f6eeb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.636775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78311232-640e-4f51-adc2-24934c2f6eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.637036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78311232-640e-4f51-adc2-24934c2f6eeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.637744 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78311232-640e-4f51-adc2-24934c2f6eeb None admin] RBAC: Adding query filter params (role.id=b81578bacade480d94c41559bc214637, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=c496da229db34f90a4b3fdcfa582dca8) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:07.638037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78311232-640e-4f51-adc2-24934c2f6eeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.639442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78311232-640e-4f51-adc2-24934c2f6eeb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.645106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 521/1048] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:52:07 2022] GET /identity/v3/role_assignments?user.id=c496da229db34f90a4b3fdcfa582dca8&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=b81578bacade480d94c41559bc214637 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:07.647469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78311232-640e-4f51-adc2-24934c2f6eeb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:07.673052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bb8ce-c5ca-432d-8f3f-1c8c83a39f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.673692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bb8ce-c5ca-432d-8f3f-1c8c83a39f0b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.673959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bb8ce-c5ca-432d-8f3f-1c8c83a39f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.674221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bb8ce-c5ca-432d-8f3f-1c8c83a39f0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.675536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bb8ce-c5ca-432d-8f3f-1c8c83a39f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.677442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bb8ce-c5ca-432d-8f3f-1c8c83a39f0b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.679341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 528/1049] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:07.685610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bb8ce-c5ca-432d-8f3f-1c8c83a39f0b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:07.701775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:07.706136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:07.764871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78311232-640e-4f51-adc2-24934c2f6eeb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.800775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bb8ce-c5ca-432d-8f3f-1c8c83a39f0b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:07.808217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e27ab-0bf5-4754-8126-32fef07511e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.808921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e27ab-0bf5-4754-8126-32fef07511e4 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.809189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e27ab-0bf5-4754-8126-32fef07511e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.809915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e27ab-0bf5-4754-8126-32fef07511e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.810572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e27ab-0bf5-4754-8126-32fef07511e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.812292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e27ab-0bf5-4754-8126-32fef07511e4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.837390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb19bd-95e0-4122-be76-c3b500360891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:07.838110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb19bd-95e0-4122-be76-c3b500360891 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:07.838452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb19bd-95e0-4122-be76-c3b500360891 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:07.838827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb19bd-95e0-4122-be76-c3b500360891 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:07.839539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb19bd-95e0-4122-be76-c3b500360891 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:07.841135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb19bd-95e0-4122-be76-c3b500360891 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:07.883186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 522/1050] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 17:52:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:52:07.929436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 529/1051] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:52:07 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:52:09.647897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da9eea-fe1b-4a5a-8540-f856ce2a75ec None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:09.648425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da9eea-fe1b-4a5a-8540-f856ce2a75ec None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:09.648425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da9eea-fe1b-4a5a-8540-f856ce2a75ec None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:09.649845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 523/1052] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:09.655628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe9de9-62ef-4815-bdeb-7a22d7865162 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:09.656054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe9de9-62ef-4815-bdeb-7a22d7865162 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:09.656201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe9de9-62ef-4815-bdeb-7a22d7865162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:09.674608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ecfe9de9-62ef-4815-bdeb-7a22d7865162 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:09.675781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ecfe9de9-62ef-4815-bdeb-7a22d7865162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:09.744651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:09.751392 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 530/1053] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:09.773764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da9eea-fe1b-4a5a-8540-f856ce2a75ec None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:09.892234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da9eea-fe1b-4a5a-8540-f856ce2a75ec None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:09.893779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:09.914960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16cb59-84a3-4dd5-884c-cb9ca476c13b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:09.915165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16cb59-84a3-4dd5-884c-cb9ca476c13b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:09.915358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16cb59-84a3-4dd5-884c-cb9ca476c13b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:09.916770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 531/1054] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:52:09.921725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821444a9-13fa-41b4-8c56-3fd7983822be None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:09.922001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821444a9-13fa-41b4-8c56-3fd7983822be None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:09.922272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821444a9-13fa-41b4-8c56-3fd7983822be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:09.930945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732e292-c2cd-4771-9f48-3308291f376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:09.931568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732e292-c2cd-4771-9f48-3308291f376c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:09.932366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732e292-c2cd-4771-9f48-3308291f376c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:09.932656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732e292-c2cd-4771-9f48-3308291f376c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:09.941588 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-821444a9-13fa-41b4-8c56-3fd7983822be None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:09.942693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732e292-c2cd-4771-9f48-3308291f376c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:09.942801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-821444a9-13fa-41b4-8c56-3fd7983822be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-8732e292-c2cd-4771-9f48-3308291f376c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:09.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:09.948786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 524/1055] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 17:52:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:09.955683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732e292-c2cd-4771-9f48-3308291f376c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.015598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 532/1056] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:52:10.016864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebef56-7858-44ab-9084-3dc70530b178 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.017171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebef56-7858-44ab-9084-3dc70530b178 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.017406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebef56-7858-44ab-9084-3dc70530b178 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.018747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 533/1057] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:10.023562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29dde7-4a7e-4918-8454-6968ac1e5013 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.024067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29dde7-4a7e-4918-8454-6968ac1e5013 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.024579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29dde7-4a7e-4918-8454-6968ac1e5013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.043925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1b29dde7-4a7e-4918-8454-6968ac1e5013 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:10.045129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1b29dde7-4a7e-4918-8454-6968ac1e5013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:10.059674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732e292-c2cd-4771-9f48-3308291f376c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.094586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92d5cb-518b-4614-bf62-607cf3f2722b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.095187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92d5cb-518b-4614-bf62-607cf3f2722b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.095459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92d5cb-518b-4614-bf62-607cf3f2722b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.095729 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92d5cb-518b-4614-bf62-607cf3f2722b None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.096413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92d5cb-518b-4614-bf62-607cf3f2722b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:10.096747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92d5cb-518b-4614-bf62-607cf3f2722b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:10.098118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92d5cb-518b-4614-bf62-607cf3f2722b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:10.106962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 525/1058] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 17:52:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:10.108081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92d5cb-518b-4614-bf62-607cf3f2722b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.117352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 534/1059] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:52:10.118193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29dde7-4a7e-4918-8454-6968ac1e5013 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.137816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:10.176428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:10.211903 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92d5cb-518b-4614-bf62-607cf3f2722b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.216943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29dde7-4a7e-4918-8454-6968ac1e5013 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.245718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8197d94-c490-4a6f-b4c1-5a490ffe75f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.246301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8197d94-c490-4a6f-b4c1-5a490ffe75f3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.246569 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8197d94-c490-4a6f-b4c1-5a490ffe75f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.246830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8197d94-c490-4a6f-b4c1-5a490ffe75f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.251718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef5efb-1d67-40b5-9b8d-ac1022da76d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.252329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef5efb-1d67-40b5-9b8d-ac1022da76d2 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.252636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef5efb-1d67-40b5-9b8d-ac1022da76d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.252904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef5efb-1d67-40b5-9b8d-ac1022da76d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.262750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef5efb-1d67-40b5-9b8d-ac1022da76d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:10.264198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef5efb-1d67-40b5-9b8d-ac1022da76d2 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-20ef5efb-1d67-40b5-9b8d-ac1022da76d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:52:10.278604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:10.279734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:52:10.280676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:52:10.280676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:10.280676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:10.280676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 535/1060] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:10.280676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef5efb-1d67-40b5-9b8d-ac1022da76d2 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.315947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8197d94-c490-4a6f-b4c1-5a490ffe75f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:10.317423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8197d94-c490-4a6f-b4c1-5a490ffe75f3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:10.340553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:10.384596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef5efb-1d67-40b5-9b8d-ac1022da76d2 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.399410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:10.424858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442da63-56bb-47eb-9655-2b8072806175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.425484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442da63-56bb-47eb-9655-2b8072806175 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.425761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442da63-56bb-47eb-9655-2b8072806175 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.426034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442da63-56bb-47eb-9655-2b8072806175 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.436928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 526/1061] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:10.438055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8197d94-c490-4a6f-b4c1-5a490ffe75f3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.440083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442da63-56bb-47eb-9655-2b8072806175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8442da63-56bb-47eb-9655-2b8072806175 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:10.445601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:10.447968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 536/1062] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:10.448889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442da63-56bb-47eb-9655-2b8072806175 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.542328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8197d94-c490-4a6f-b4c1-5a490ffe75f3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.548098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442da63-56bb-47eb-9655-2b8072806175 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.567210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:10.577916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f62da7-714b-4fa8-9ba1-6591257b26f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.578770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f62da7-714b-4fa8-9ba1-6591257b26f5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.584364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f62da7-714b-4fa8-9ba1-6591257b26f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.584364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f62da7-714b-4fa8-9ba1-6591257b26f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.584364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f62da7-714b-4fa8-9ba1-6591257b26f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:10.584364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f62da7-714b-4fa8-9ba1-6591257b26f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:10.584734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d227838-ce31-4ed3-b31c-b5e7d293bf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.584832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f62da7-714b-4fa8-9ba1-6591257b26f5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:10.584906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d227838-ce31-4ed3-b31c-b5e7d293bf54 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.584906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d227838-ce31-4ed3-b31c-b5e7d293bf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.584906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d227838-ce31-4ed3-b31c-b5e7d293bf54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.593079 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d227838-ce31-4ed3-b31c-b5e7d293bf54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:10.593492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 527/1063] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:10.594271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f62da7-714b-4fa8-9ba1-6591257b26f5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.594510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d227838-ce31-4ed3-b31c-b5e7d293bf54 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-8d227838-ce31-4ed3-b31c-b5e7d293bf54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:52:10.611606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:10.617202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:52:10.618198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:52:10.618198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:10.618198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:10.618198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 537/1064] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:10.618198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d227838-ce31-4ed3-b31c-b5e7d293bf54 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.642330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:10.696519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f62da7-714b-4fa8-9ba1-6591257b26f5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.717721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d227838-ce31-4ed3-b31c-b5e7d293bf54 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.730639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb205b-9148-4b25-b73f-78de95945976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.731250 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb205b-9148-4b25-b73f-78de95945976 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.731517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb205b-9148-4b25-b73f-78de95945976 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.731783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb205b-9148-4b25-b73f-78de95945976 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.732431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb205b-9148-4b25-b73f-78de95945976 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:10.732755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb205b-9148-4b25-b73f-78de95945976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:10.734105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb205b-9148-4b25-b73f-78de95945976 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:10.742384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 528/1065] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:10.743155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb205b-9148-4b25-b73f-78de95945976 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.752497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2684465-afad-4e0e-9020-84339eeecf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.753150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2684465-afad-4e0e-9020-84339eeecf75 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.753425 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2684465-afad-4e0e-9020-84339eeecf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.753690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2684465-afad-4e0e-9020-84339eeecf75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.761378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2684465-afad-4e0e-9020-84339eeecf75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:10.763040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2684465-afad-4e0e-9020-84339eeecf75 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d2684465-afad-4e0e-9020-84339eeecf75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:52:10.774298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:10.775256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 538/1066] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:10.782642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2684465-afad-4e0e-9020-84339eeecf75 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.793061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:10.876173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb205b-9148-4b25-b73f-78de95945976 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.892177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:10.910268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891d586-b079-4992-b8c9-67cf727e6afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.910854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891d586-b079-4992-b8c9-67cf727e6afa None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.911121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891d586-b079-4992-b8c9-67cf727e6afa None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.911381 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891d586-b079-4992-b8c9-67cf727e6afa None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.912004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891d586-b079-4992-b8c9-67cf727e6afa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:10.912295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891d586-b079-4992-b8c9-67cf727e6afa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:10.912624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2684465-afad-4e0e-9020-84339eeecf75 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:10.913748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891d586-b079-4992-b8c9-67cf727e6afa None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:10.925644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 529/1067] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:10.926623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891d586-b079-4992-b8c9-67cf727e6afa None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:10.948914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118e80e-9b11-440a-acec-45d24c4c6134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:10.949469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118e80e-9b11-440a-acec-45d24c4c6134 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:10.949747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118e80e-9b11-440a-acec-45d24c4c6134 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:10.950012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118e80e-9b11-440a-acec-45d24c4c6134 None admin] PATH_INFO: `/v3/users/ded308bab15247a2ac95e932ec46505f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:10.966988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118e80e-9b11-440a-acec-45d24c4c6134 None admin] RBAC: Authorizing `identity:get_user(user_id=ded308bab15247a2ac95e932ec46505f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:10.968598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118e80e-9b11-440a-acec-45d24c4c6134 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:10.975435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 539/1068] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/users/ded308bab15247a2ac95e932ec46505f => generated 285 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:10.976451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118e80e-9b11-440a-acec-45d24c4c6134 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:11.042897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891d586-b079-4992-b8c9-67cf727e6afa None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:11.050471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:11.078631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bbb41-85c1-4d7f-94f3-e6bc9184b8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:11.079243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bbb41-85c1-4d7f-94f3-e6bc9184b8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:11.079526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bbb41-85c1-4d7f-94f3-e6bc9184b8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:11.079781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bbb41-85c1-4d7f-94f3-e6bc9184b8e1 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:11.080531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bbb41-85c1-4d7f-94f3-e6bc9184b8e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:11.080877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bbb41-85c1-4d7f-94f3-e6bc9184b8e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:11.082342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bbb41-85c1-4d7f-94f3-e6bc9184b8e1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:11.090853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118e80e-9b11-440a-acec-45d24c4c6134 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:11.102396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 530/1069] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:11.107532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bbb41-85c1-4d7f-94f3-e6bc9184b8e1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:11.133255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b397539-1b81-459a-b899-11f9cea8da5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:11.133860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b397539-1b81-459a-b899-11f9cea8da5c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:11.134134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b397539-1b81-459a-b899-11f9cea8da5c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:11.134403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b397539-1b81-459a-b899-11f9cea8da5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:11.146350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:11.219145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b397539-1b81-459a-b899-11f9cea8da5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:11.219145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b397539-1b81-459a-b899-11f9cea8da5c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:11.220809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bbb41-85c1-4d7f-94f3-e6bc9184b8e1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:11.255042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690b2c7-393d-4c9c-a5ad-a243f05bac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:11.255653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690b2c7-393d-4c9c-a5ad-a243f05bac14 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:11.255928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690b2c7-393d-4c9c-a5ad-a243f05bac14 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:11.256196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690b2c7-393d-4c9c-a5ad-a243f05bac14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:11.267080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690b2c7-393d-4c9c-a5ad-a243f05bac14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:11.268557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690b2c7-393d-4c9c-a5ad-a243f05bac14 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3690b2c7-393d-4c9c-a5ad-a243f05bac14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:52:11.283158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:11.284301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:52:11.285343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:52:11.285343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:11.285343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:11.285343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 531/1070] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:52:11.289857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690b2c7-393d-4c9c-a5ad-a243f05bac14 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:11.308515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:11.318832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 540/1071] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:11.325088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b397539-1b81-459a-b899-11f9cea8da5c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:11.365542 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:11.391884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690b2c7-393d-4c9c-a5ad-a243f05bac14 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:11.421301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b397539-1b81-459a-b899-11f9cea8da5c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:11.425580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24c960-9157-4580-8184-8e644edf71a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:11.426173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24c960-9157-4580-8184-8e644edf71a8 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:11.426447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24c960-9157-4580-8184-8e644edf71a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:11.426715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24c960-9157-4580-8184-8e644edf71a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:11.427343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24c960-9157-4580-8184-8e644edf71a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:11.427637 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24c960-9157-4580-8184-8e644edf71a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:11.429023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24c960-9157-4580-8184-8e644edf71a8 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:11.441204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 532/1072] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:11.446815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24c960-9157-4580-8184-8e644edf71a8 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:11.456633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9715a-545d-4db2-86be-c63b00e7f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:11.457247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9715a-545d-4db2-86be-c63b00e7f250 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:11.457518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9715a-545d-4db2-86be-c63b00e7f250 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:11.457782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9715a-545d-4db2-86be-c63b00e7f250 None admin] PATH_INFO: `/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:11.458875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9715a-545d-4db2-86be-c63b00e7f250 None admin] RBAC: Authorizing `identity:get_project(project_id=ab74c2ec77cd4eedaa75ad690b7611cf)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:11.460284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9715a-545d-4db2-86be-c63b00e7f250 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:11.461902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 541/1073] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:11 2022] GET /identity/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:11.467723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9715a-545d-4db2-86be-c63b00e7f250 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:11.521502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:11.543193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24c960-9157-4580-8184-8e644edf71a8 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:11.563930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9715a-545d-4db2-86be-c63b00e7f250 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:11.573797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:11.577193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b10571-8ae6-4bf0-b0a0-b037b9019bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:11.577815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b10571-8ae6-4bf0-b0a0-b037b9019bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:11.578335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b10571-8ae6-4bf0-b0a0-b037b9019bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:11.578605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b10571-8ae6-4bf0-b0a0-b037b9019bd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:11.585173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b10571-8ae6-4bf0-b0a0-b037b9019bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:11.586604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b10571-8ae6-4bf0-b0a0-b037b9019bd4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-87b10571-8ae6-4bf0-b0a0-b037b9019bd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:52:11.597782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:11.598824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:11.600167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:52:11.600167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:52:11.600167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:52:11.600167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:52:11.600167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:52:11.600167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:52:11.600167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:52:11.600167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:11.600167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 533/1074] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:52:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:52:11.600769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946203fc-bd89-4a30-b101-77efefe09793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:11.600769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946203fc-bd89-4a30-b101-77efefe09793 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:11.600769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946203fc-bd89-4a30-b101-77efefe09793 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:11.600976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946203fc-bd89-4a30-b101-77efefe09793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:11.601631 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946203fc-bd89-4a30-b101-77efefe09793 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=ab74c2ec77cd4eedaa75ad690b7611cf, user.id=ded308bab15247a2ac95e932ec46505f) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:11.601930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946203fc-bd89-4a30-b101-77efefe09793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:11.603326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946203fc-bd89-4a30-b101-77efefe09793 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:11.604625 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b10571-8ae6-4bf0-b0a0-b037b9019bd4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:11.624534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 542/1075] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:52:11 2022] GET /identity/v3/role_assignments?user.id=ded308bab15247a2ac95e932ec46505f&scope.project.id=ab74c2ec77cd4eedaa75ad690b7611cf&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:11.707935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b10571-8ae6-4bf0-b0a0-b037b9019bd4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:11.743469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d01d51-e103-47de-9d79-f4e7b3a07bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:11.744774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d01d51-e103-47de-9d79-f4e7b3a07bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:11.745075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d01d51-e103-47de-9d79-f4e7b3a07bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:11.745341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d01d51-e103-47de-9d79-f4e7b3a07bd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:11.746033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d01d51-e103-47de-9d79-f4e7b3a07bd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:11.746329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d01d51-e103-47de-9d79-f4e7b3a07bd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:11.747765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d01d51-e103-47de-9d79-f4e7b3a07bd4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:11.750417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:11.759922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 534/1076] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:52:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:11.766444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946203fc-bd89-4a30-b101-77efefe09793 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:11.869829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946203fc-bd89-4a30-b101-77efefe09793 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:11.903520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbeb0f-564d-46b6-9b94-6be541159133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:11.904174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbeb0f-564d-46b6-9b94-6be541159133 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:11.904447 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbeb0f-564d-46b6-9b94-6be541159133 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:11.904748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbeb0f-564d-46b6-9b94-6be541159133 None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/c496da229db34f90a4b3fdcfa582dca8/roles/b81578bacade480d94c41559bc214637` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:11.906907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbeb0f-564d-46b6-9b94-6be541159133 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=c496da229db34f90a4b3fdcfa582dca8, role_id=b81578bacade480d94c41559bc214637)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:11.908340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbeb0f-564d-46b6-9b94-6be541159133 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:11.971591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 543/1077] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:52:11 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/c496da229db34f90a4b3fdcfa582dca8/roles/b81578bacade480d94c41559bc214637 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:52:13.590526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fd445-294e-4b94-8977-4d0672222bc7 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:13.591052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fd445-294e-4b94-8977-4d0672222bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:13.591052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fd445-294e-4b94-8977-4d0672222bc7 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:13.592376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 535/1078] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:13.597564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c20ed-f55f-4a37-8df3-5eb0dc5e9923 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:13.597968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c20ed-f55f-4a37-8df3-5eb0dc5e9923 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:13.598105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c20ed-f55f-4a37-8df3-5eb0dc5e9923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:13.616214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-274c20ed-f55f-4a37-8df3-5eb0dc5e9923 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:13.617404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-274c20ed-f55f-4a37-8df3-5eb0dc5e9923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:13.642138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:13.682241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 544/1079] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:13.704961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fd445-294e-4b94-8977-4d0672222bc7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:13.811161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fd445-294e-4b94-8977-4d0672222bc7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:13.852851 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67206df-3ade-49c0-abaf-5adb6adc17de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:13.853430 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67206df-3ade-49c0-abaf-5adb6adc17de None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:13.854229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67206df-3ade-49c0-abaf-5adb6adc17de None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:13.854515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67206df-3ade-49c0-abaf-5adb6adc17de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:13.864778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67206df-3ade-49c0-abaf-5adb6adc17de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f67206df-3ade-49c0-abaf-5adb6adc17de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:13.869630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:13.870697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:13.871553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 536/1080] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:52:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:13.876779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c20ed-f55f-4a37-8df3-5eb0dc5e9923 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:13.946663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed5977-8c0a-40e1-bf4a-077e3f20daa4 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:13.946940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed5977-8c0a-40e1-bf4a-077e3f20daa4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:13.947714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed5977-8c0a-40e1-bf4a-077e3f20daa4 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:13.949121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 537/1081] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:52:13.954211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273332a5-e16e-4867-a091-178a78132c0f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:13.954486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273332a5-e16e-4867-a091-178a78132c0f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:13.954757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273332a5-e16e-4867-a091-178a78132c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:13.972760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-273332a5-e16e-4867-a091-178a78132c0f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:13.973919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-273332a5-e16e-4867-a091-178a78132c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:13.974534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c20ed-f55f-4a37-8df3-5eb0dc5e9923 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:14.007790 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea5bed-168f-4617-9ddd-3a0ef3696efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:14.008416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea5bed-168f-4617-9ddd-3a0ef3696efe None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:14.008732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea5bed-168f-4617-9ddd-3a0ef3696efe None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:14.008999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea5bed-168f-4617-9ddd-3a0ef3696efe None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:14.010151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea5bed-168f-4617-9ddd-3a0ef3696efe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:14.010445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea5bed-168f-4617-9ddd-3a0ef3696efe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:14.011805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea5bed-168f-4617-9ddd-3a0ef3696efe None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:14.019864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 545/1082] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:52:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:14.026138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea5bed-168f-4617-9ddd-3a0ef3696efe None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:14.033331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:14.051184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:14.051712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 538/1083] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:52:14.076213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273332a5-e16e-4867-a091-178a78132c0f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:14.124760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea5bed-168f-4617-9ddd-3a0ef3696efe None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:14.158302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf42b6-0a6f-4f9d-9ff3-1e29fa203bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:14.158910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf42b6-0a6f-4f9d-9ff3-1e29fa203bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:14.159182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf42b6-0a6f-4f9d-9ff3-1e29fa203bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:14.159454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf42b6-0a6f-4f9d-9ff3-1e29fa203bc9 None admin] PATH_INFO: `/v3/users/ded308bab15247a2ac95e932ec46505f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:14.160621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf42b6-0a6f-4f9d-9ff3-1e29fa203bc9 None admin] RBAC: Authorizing `identity:get_user(user_id=ded308bab15247a2ac95e932ec46505f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:14.162057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf42b6-0a6f-4f9d-9ff3-1e29fa203bc9 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:14.163786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 546/1084] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:14 2022] GET /identity/v3/users/ded308bab15247a2ac95e932ec46505f => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:14.169346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf42b6-0a6f-4f9d-9ff3-1e29fa203bc9 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:14.176480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273332a5-e16e-4867-a091-178a78132c0f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:14.214314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b042fb5-8c07-4140-8c43-bb57f732bf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:14.214932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b042fb5-8c07-4140-8c43-bb57f732bf5b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:14.215201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b042fb5-8c07-4140-8c43-bb57f732bf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:14.215472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b042fb5-8c07-4140-8c43-bb57f732bf5b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:14.225346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b042fb5-8c07-4140-8c43-bb57f732bf5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-0b042fb5-8c07-4140-8c43-bb57f732bf5b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:14.230031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:14.230963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:14.231820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 539/1085] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 17:52:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:52:14.237122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b042fb5-8c07-4140-8c43-bb57f732bf5b None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:14.261952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:14.270321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf42b6-0a6f-4f9d-9ff3-1e29fa203bc9 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:14.284709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:14.303478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19778a75-befd-4f29-b703-e17058663cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:14.304096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19778a75-befd-4f29-b703-e17058663cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:14.304371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19778a75-befd-4f29-b703-e17058663cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:14.304683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19778a75-befd-4f29-b703-e17058663cd6 None admin] PATH_INFO: `/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:14.305793 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19778a75-befd-4f29-b703-e17058663cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=ab74c2ec77cd4eedaa75ad690b7611cf)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:14.307287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19778a75-befd-4f29-b703-e17058663cd6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:14.308948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 547/1086] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:14 2022] GET /identity/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:14.314679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19778a75-befd-4f29-b703-e17058663cd6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:14.336450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b042fb5-8c07-4140-8c43-bb57f732bf5b None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:14.371021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0d45-9cca-436a-86cc-4712d3e85f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:14.371606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0d45-9cca-436a-86cc-4712d3e85f72 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:14.371883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0d45-9cca-436a-86cc-4712d3e85f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:14.372150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0d45-9cca-436a-86cc-4712d3e85f72 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:14.372833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e0d45-9cca-436a-86cc-4712d3e85f72 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:14.373127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e0d45-9cca-436a-86cc-4712d3e85f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:14.374491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e0d45-9cca-436a-86cc-4712d3e85f72 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:14.384362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 540/1087] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 17:52:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:14.390577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0d45-9cca-436a-86cc-4712d3e85f72 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:14.412785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19778a75-befd-4f29-b703-e17058663cd6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:14.445270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53439af-e9ba-4b04-bb6f-98b0f878940c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:14.445892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53439af-e9ba-4b04-bb6f-98b0f878940c None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:14.446161 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53439af-e9ba-4b04-bb6f-98b0f878940c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:14.446426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53439af-e9ba-4b04-bb6f-98b0f878940c None admin] PATH_INFO: `/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf/users/ded308bab15247a2ac95e932ec46505f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:14.449020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53439af-e9ba-4b04-bb6f-98b0f878940c None admin] RBAC: Authorizing `identity:create_grant(project_id=ab74c2ec77cd4eedaa75ad690b7611cf, user_id=ded308bab15247a2ac95e932ec46505f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:14.450438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53439af-e9ba-4b04-bb6f-98b0f878940c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:14.493213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0d45-9cca-436a-86cc-4712d3e85f72 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:14.502350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:14.525904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 548/1088] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:52:14 2022] PUT /identity/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf/users/ded308bab15247a2ac95e932ec46505f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:52:14.526347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8230cc7-ed98-490b-9d5b-bcbdbdf22062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:14.526960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8230cc7-ed98-490b-9d5b-bcbdbdf22062 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:14.527229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8230cc7-ed98-490b-9d5b-bcbdbdf22062 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:14.527490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8230cc7-ed98-490b-9d5b-bcbdbdf22062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:14.538907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8230cc7-ed98-490b-9d5b-bcbdbdf22062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:14.540383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8230cc7-ed98-490b-9d5b-bcbdbdf22062 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-f8230cc7-ed98-490b-9d5b-bcbdbdf22062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:52:14.555922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:14.559585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:52:14.560557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:52:14.560557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:14.560557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:14.560557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 541/1089] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:52:14.563835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53439af-e9ba-4b04-bb6f-98b0f878940c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:14.589210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:14.669929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53439af-e9ba-4b04-bb6f-98b0f878940c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:14.706647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cbb09-9aff-4891-850f-4f20432a662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:14.707246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cbb09-9aff-4891-850f-4f20432a662f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:14.707520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cbb09-9aff-4891-850f-4f20432a662f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:14.707786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cbb09-9aff-4891-850f-4f20432a662f None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:14.708413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cbb09-9aff-4891-850f-4f20432a662f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:14.708737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cbb09-9aff-4891-850f-4f20432a662f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:14.710081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cbb09-9aff-4891-850f-4f20432a662f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:14.721795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 549/1090] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:14.727523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8230cc7-ed98-490b-9d5b-bcbdbdf22062 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:14.830708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8230cc7-ed98-490b-9d5b-bcbdbdf22062 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:14.865554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd18c6-e054-47d7-b7fb-5d8785ab65b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:14.866171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd18c6-e054-47d7-b7fb-5d8785ab65b1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:14.866440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd18c6-e054-47d7-b7fb-5d8785ab65b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:14.866702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd18c6-e054-47d7-b7fb-5d8785ab65b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:14.876927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd18c6-e054-47d7-b7fb-5d8785ab65b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:14.878352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd18c6-e054-47d7-b7fb-5d8785ab65b1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-4acd18c6-e054-47d7-b7fb-5d8785ab65b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:52:14.889131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:14.890291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:14.891278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 542/1091] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 17:52:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:14.895909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cbb09-9aff-4891-850f-4f20432a662f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:14.977213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:14.996259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cbb09-9aff-4891-850f-4f20432a662f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:15.029359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47ef42-3429-4d93-ae6d-b9929fc13313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:15.029959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47ef42-3429-4d93-ae6d-b9929fc13313 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:15.030233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47ef42-3429-4d93-ae6d-b9929fc13313 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:15.030499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47ef42-3429-4d93-ae6d-b9929fc13313 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:15.031215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47ef42-3429-4d93-ae6d-b9929fc13313 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:15.031516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47ef42-3429-4d93-ae6d-b9929fc13313 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:15.032974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47ef42-3429-4d93-ae6d-b9929fc13313 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:15.050093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 550/1092] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 17:52:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:15.056040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd18c6-e054-47d7-b7fb-5d8785ab65b1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:15.079248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:15.155381 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd18c6-e054-47d7-b7fb-5d8785ab65b1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:15.188360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17810cae-7f3d-468e-a142-f362e89dca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:15.188985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17810cae-7f3d-468e-a142-f362e89dca8c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:15.189262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17810cae-7f3d-468e-a142-f362e89dca8c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:15.189533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17810cae-7f3d-468e-a142-f362e89dca8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:15.254701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17810cae-7f3d-468e-a142-f362e89dca8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:15.256066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17810cae-7f3d-468e-a142-f362e89dca8c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:15.276114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:15.350777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 543/1093] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:15.357097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47ef42-3429-4d93-ae6d-b9929fc13313 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:15.452198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47ef42-3429-4d93-ae6d-b9929fc13313 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:15.486077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25351d5-c0e8-4e66-8794-b8bb3e0a343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:15.486690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25351d5-c0e8-4e66-8794-b8bb3e0a343f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:15.486965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25351d5-c0e8-4e66-8794-b8bb3e0a343f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:15.487231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25351d5-c0e8-4e66-8794-b8bb3e0a343f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:15.493625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:15.497265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25351d5-c0e8-4e66-8794-b8bb3e0a343f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:15.498708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25351d5-c0e8-4e66-8794-b8bb3e0a343f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-a25351d5-c0e8-4e66-8794-b8bb3e0a343f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:52:15.519042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:15.520106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:52:15.521141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:52:15.521141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:52:15.521141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:15.521141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 551/1094] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:52:15.521418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17810cae-7f3d-468e-a142-f362e89dca8c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:15.618527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17810cae-7f3d-468e-a142-f362e89dca8c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:15.639537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:15.654047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f8290-870d-46a6-aca6-8a3427011bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:15.658050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f8290-870d-46a6-aca6-8a3427011bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:15.658050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f8290-870d-46a6-aca6-8a3427011bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:15.658050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f8290-870d-46a6-aca6-8a3427011bb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:15.658050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f8290-870d-46a6-aca6-8a3427011bb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:15.658050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f8290-870d-46a6-aca6-8a3427011bb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:15.658050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f8290-870d-46a6-aca6-8a3427011bb3 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:15.668127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 544/1095] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:52:15.673856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25351d5-c0e8-4e66-8794-b8bb3e0a343f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:15.770414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25351d5-c0e8-4e66-8794-b8bb3e0a343f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:15.804295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9aa0b-e1ef-485c-9581-44c828fe31f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:15.804938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9aa0b-e1ef-485c-9581-44c828fe31f6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:15.805216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9aa0b-e1ef-485c-9581-44c828fe31f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:15.805482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9aa0b-e1ef-485c-9581-44c828fe31f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:15.811962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9aa0b-e1ef-485c-9581-44c828fe31f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:15.813409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9aa0b-e1ef-485c-9581-44c828fe31f6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b4d9aa0b-e1ef-485c-9581-44c828fe31f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:52:15.824743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:15.825773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:15.826688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:52:15.826688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:52:15.826688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:52:15.826688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:52:15.826688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:52:15.826688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:52:15.826688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:52:15.826688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:15.826688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 552/1096] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:52:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:15.831945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f8290-870d-46a6-aca6-8a3427011bb3 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:15.927590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f8290-870d-46a6-aca6-8a3427011bb3 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:15.960826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d143584-8b1a-499f-972d-ca42af0c4a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:15.961414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d143584-8b1a-499f-972d-ca42af0c4a71 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:15.961683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d143584-8b1a-499f-972d-ca42af0c4a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:15.961944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d143584-8b1a-499f-972d-ca42af0c4a71 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:15.964491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d143584-8b1a-499f-972d-ca42af0c4a71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:15.964834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d143584-8b1a-499f-972d-ca42af0c4a71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:15.966256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d143584-8b1a-499f-972d-ca42af0c4a71 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:15.978130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 545/1097] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:52:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:15.984166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9aa0b-e1ef-485c-9581-44c828fe31f6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:16.032655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:16.081062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9aa0b-e1ef-485c-9581-44c828fe31f6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:16.114225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005caa8c-7a3c-463e-a9a4-9ebe57f4ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:16.114808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005caa8c-7a3c-463e-a9a4-9ebe57f4ca45 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:16.115075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005caa8c-7a3c-463e-a9a4-9ebe57f4ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:16.115333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005caa8c-7a3c-463e-a9a4-9ebe57f4ca45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:16.116031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005caa8c-7a3c-463e-a9a4-9ebe57f4ca45 None admin] RBAC: Adding query filter params (role.id=b81578bacade480d94c41559bc214637, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=c496da229db34f90a4b3fdcfa582dca8) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:16.116324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005caa8c-7a3c-463e-a9a4-9ebe57f4ca45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:16.117754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005caa8c-7a3c-463e-a9a4-9ebe57f4ca45 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:16.123991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 553/1098] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:52:15 2022] GET /identity/v3/role_assignments?user.id=c496da229db34f90a4b3fdcfa582dca8&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=b81578bacade480d94c41559bc214637 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:16.520781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6084785-557c-48e2-8910-ffd1ddf83749 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:16.521069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6084785-557c-48e2-8910-ffd1ddf83749 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:16.521338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6084785-557c-48e2-8910-ffd1ddf83749 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:16.522680 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 546/1099] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:16.527935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566132b7-6e0b-4790-8640-d0549585cd44 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:16.528223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566132b7-6e0b-4790-8640-d0549585cd44 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:16.528495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566132b7-6e0b-4790-8640-d0549585cd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:16.547314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-566132b7-6e0b-4790-8640-d0549585cd44 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:16.548564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-566132b7-6e0b-4790-8640-d0549585cd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:16.616366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 554/1100] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:16.639344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6084785-557c-48e2-8910-ffd1ddf83749 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:16.684129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:16.744111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6084785-557c-48e2-8910-ffd1ddf83749 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:16.778534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaec5d-68ab-4cac-9a2c-a349cfb390bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:16.779088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaec5d-68ab-4cac-9a2c-a349cfb390bb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:16.779363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaec5d-68ab-4cac-9a2c-a349cfb390bb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:16.779632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaec5d-68ab-4cac-9a2c-a349cfb390bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:16.790232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faaec5d-68ab-4cac-9a2c-a349cfb390bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-7faaec5d-68ab-4cac-9a2c-a349cfb390bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:16.795650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:16.796666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:16.797524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 547/1101] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:52:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:16.803241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566132b7-6e0b-4790-8640-d0549585cd44 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:16.851291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:16.910133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566132b7-6e0b-4790-8640-d0549585cd44 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:16.944270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdcb8d-abe5-4f66-9938-0a5953006965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:16.944895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdcb8d-abe5-4f66-9938-0a5953006965 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:16.945176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdcb8d-abe5-4f66-9938-0a5953006965 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:16.945442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdcb8d-abe5-4f66-9938-0a5953006965 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:16.946098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcdcb8d-abe5-4f66-9938-0a5953006965 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:16.946393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcdcb8d-abe5-4f66-9938-0a5953006965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:16.948276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcdcb8d-abe5-4f66-9938-0a5953006965 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:16.956797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 555/1102] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:52:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:16.963408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaec5d-68ab-4cac-9a2c-a349cfb390bb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:17.058057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:17.060988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaec5d-68ab-4cac-9a2c-a349cfb390bb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:17.094544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89cbec-d670-4bff-bedb-838a165b272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:17.095152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89cbec-d670-4bff-bedb-838a165b272c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:17.095427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89cbec-d670-4bff-bedb-838a165b272c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:17.095696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89cbec-d670-4bff-bedb-838a165b272c None admin] PATH_INFO: `/v3/users/ded308bab15247a2ac95e932ec46505f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:17.096832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89cbec-d670-4bff-bedb-838a165b272c None admin] RBAC: Authorizing `identity:get_user(user_id=ded308bab15247a2ac95e932ec46505f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:17.098249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89cbec-d670-4bff-bedb-838a165b272c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:17.099800 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 548/1103] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:16 2022] GET /identity/v3/users/ded308bab15247a2ac95e932ec46505f => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:17.105721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdcb8d-abe5-4f66-9938-0a5953006965 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:17.207847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdcb8d-abe5-4f66-9938-0a5953006965 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:17.215132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:17.240971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e4c21-5bf2-4590-b022-57ddd6518a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:17.241559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e4c21-5bf2-4590-b022-57ddd6518a0b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:17.241834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e4c21-5bf2-4590-b022-57ddd6518a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:17.242102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e4c21-5bf2-4590-b022-57ddd6518a0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:17.306473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e4c21-5bf2-4590-b022-57ddd6518a0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:17.307901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e4c21-5bf2-4590-b022-57ddd6518a0b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:17.402389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 556/1104] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:17.408334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89cbec-d670-4bff-bedb-838a165b272c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:17.502971 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89cbec-d670-4bff-bedb-838a165b272c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:17.534683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237da2c0-0d87-49e5-834c-e6a0be5bfbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:17.535278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237da2c0-0d87-49e5-834c-e6a0be5bfbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:17.535546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237da2c0-0d87-49e5-834c-e6a0be5bfbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:17.535807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237da2c0-0d87-49e5-834c-e6a0be5bfbf0 None admin] PATH_INFO: `/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:17.536911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237da2c0-0d87-49e5-834c-e6a0be5bfbf0 None admin] RBAC: Authorizing `identity:get_project(project_id=ab74c2ec77cd4eedaa75ad690b7611cf)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:17.538308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237da2c0-0d87-49e5-834c-e6a0be5bfbf0 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:17.539858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 549/1105] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:17 2022] GET /identity/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:17.545699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e4c21-5bf2-4590-b022-57ddd6518a0b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:17.553477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:17.642468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e4c21-5bf2-4590-b022-57ddd6518a0b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:17.674308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a1ce5-dc37-470e-ac02-a72211e0a6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:17.674836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a1ce5-dc37-470e-ac02-a72211e0a6eb None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:17.675106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a1ce5-dc37-470e-ac02-a72211e0a6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:17.675375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a1ce5-dc37-470e-ac02-a72211e0a6eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:17.676085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a1ce5-dc37-470e-ac02-a72211e0a6eb None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=ab74c2ec77cd4eedaa75ad690b7611cf, user.id=ded308bab15247a2ac95e932ec46505f) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:17.676384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a1ce5-dc37-470e-ac02-a72211e0a6eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:17.677827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a1ce5-dc37-470e-ac02-a72211e0a6eb None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:17.683919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 557/1106] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:52:17 2022] GET /identity/v3/role_assignments?user.id=ded308bab15247a2ac95e932ec46505f&scope.project.id=ab74c2ec77cd4eedaa75ad690b7611cf&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:19.703855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba55b93-dc3d-4655-836c-a8f252b04184 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:19.704404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba55b93-dc3d-4655-836c-a8f252b04184 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:19.704404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba55b93-dc3d-4655-836c-a8f252b04184 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:19.705755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 550/1107] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:19.710770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2243cf-c3ed-44f8-b604-11327dd47b23 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:19.711181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2243cf-c3ed-44f8-b604-11327dd47b23 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:19.711304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2243cf-c3ed-44f8-b604-11327dd47b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:19.730085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bf2243cf-c3ed-44f8-b604-11327dd47b23 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:19.731233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bf2243cf-c3ed-44f8-b604-11327dd47b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:19.789737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:19.794671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 558/1108] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:19.818289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba55b93-dc3d-4655-836c-a8f252b04184 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:19.855048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:19.917688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba55b93-dc3d-4655-836c-a8f252b04184 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:19.949939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fd74e-cae5-4611-be40-6498d054a284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:19.950539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fd74e-cae5-4611-be40-6498d054a284 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:19.950815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fd74e-cae5-4611-be40-6498d054a284 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:19.951082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fd74e-cae5-4611-be40-6498d054a284 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:19.952219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fd74e-cae5-4611-be40-6498d054a284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:19.953977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fd74e-cae5-4611-be40-6498d054a284 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:19.955460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 551/1109] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:19.961203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2243cf-c3ed-44f8-b604-11327dd47b23 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:20.055734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2243cf-c3ed-44f8-b604-11327dd47b23 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:20.088771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4096bf-03d4-49f2-b21e-b3d055e92bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:20.089352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4096bf-03d4-49f2-b21e-b3d055e92bcd None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:20.089624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4096bf-03d4-49f2-b21e-b3d055e92bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:20.089888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4096bf-03d4-49f2-b21e-b3d055e92bcd None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:20.090514 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4096bf-03d4-49f2-b21e-b3d055e92bcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:20.091983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4096bf-03d4-49f2-b21e-b3d055e92bcd None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:20.177847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 559/1110] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:52:19 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:52:22.217131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed31d23-c11e-4a03-bf41-99c45948810a None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:22.217642 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed31d23-c11e-4a03-bf41-99c45948810a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:22.217642 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed31d23-c11e-4a03-bf41-99c45948810a None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:22.218919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 552/1111] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:22.224076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc693f0-0ae7-44d8-8797-c1ab77752cb1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:22.224527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc693f0-0ae7-44d8-8797-c1ab77752cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:22.224653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc693f0-0ae7-44d8-8797-c1ab77752cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:22.242677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ddc693f0-0ae7-44d8-8797-c1ab77752cb1 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:22.244464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ddc693f0-0ae7-44d8-8797-c1ab77752cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:22.292719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:22.308862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 560/1112] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:52:22.332666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed31d23-c11e-4a03-bf41-99c45948810a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:22.429528 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed31d23-c11e-4a03-bf41-99c45948810a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:22.432320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:22.462353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c10ed-9217-4a30-966a-e7c7e6226265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:22.462945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c10ed-9217-4a30-966a-e7c7e6226265 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:22.463215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c10ed-9217-4a30-966a-e7c7e6226265 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:22.463475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c10ed-9217-4a30-966a-e7c7e6226265 None admin] PATH_INFO: `/v3/roles/b2ded61baed84402a2fa33eda9462c6f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:22.464856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c10ed-9217-4a30-966a-e7c7e6226265 None admin] RBAC: Authorizing `identity:get_role(role_id=b2ded61baed84402a2fa33eda9462c6f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:22.466203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c10ed-9217-4a30-966a-e7c7e6226265 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:22.467536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 553/1113] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:22 2022] GET /identity/v3/roles/b2ded61baed84402a2fa33eda9462c6f => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:22.473439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc693f0-0ae7-44d8-8797-c1ab77752cb1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:22.571501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc693f0-0ae7-44d8-8797-c1ab77752cb1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:22.604962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa1372-68c4-438f-bcbe-c8c592be1415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:22.605565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa1372-68c4-438f-bcbe-c8c592be1415 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:22.605832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa1372-68c4-438f-bcbe-c8c592be1415 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:22.606092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa1372-68c4-438f-bcbe-c8c592be1415 None admin] PATH_INFO: `/v3/users/798f01254a3447e7a935dafb3ec4a5b3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:22.618723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa1372-68c4-438f-bcbe-c8c592be1415 None admin] RBAC: Authorizing `identity:get_user(user_id=798f01254a3447e7a935dafb3ec4a5b3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:22.620142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa1372-68c4-438f-bcbe-c8c592be1415 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:22.621784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 561/1114] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:22 2022] GET /identity/v3/users/798f01254a3447e7a935dafb3ec4a5b3 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:22.627592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c10ed-9217-4a30-966a-e7c7e6226265 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:22.721637 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c10ed-9217-4a30-966a-e7c7e6226265 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:22.753459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabe5b6-1316-43d3-b096-228d6d87312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:22.754030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabe5b6-1316-43d3-b096-228d6d87312a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:22.754297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabe5b6-1316-43d3-b096-228d6d87312a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:22.754557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabe5b6-1316-43d3-b096-228d6d87312a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:22.793128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:22.817983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabe5b6-1316-43d3-b096-228d6d87312a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:22.819412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabe5b6-1316-43d3-b096-228d6d87312a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:22.922845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 554/1115] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:22.928819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa1372-68c4-438f-bcbe-c8c592be1415 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:22.974323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:23.026643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa1372-68c4-438f-bcbe-c8c592be1415 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:23.058862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03f055-2963-485c-b738-a30f763c0512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:23.059468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03f055-2963-485c-b738-a30f763c0512 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:23.059744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03f055-2963-485c-b738-a30f763c0512 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:23.060013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03f055-2963-485c-b738-a30f763c0512 None admin] PATH_INFO: `/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:23.061123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03f055-2963-485c-b738-a30f763c0512 None admin] RBAC: Authorizing `identity:get_project(project_id=ab74c2ec77cd4eedaa75ad690b7611cf)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:23.062563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03f055-2963-485c-b738-a30f763c0512 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:23.064167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 562/1116] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:22 2022] GET /identity/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:23.070242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabe5b6-1316-43d3-b096-228d6d87312a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:23.137486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:23.165265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabe5b6-1316-43d3-b096-228d6d87312a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:23.197891 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a6179-5a07-4e30-b84b-4599a9009c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:23.198474 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a6179-5a07-4e30-b84b-4599a9009c53 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:23.198747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a6179-5a07-4e30-b84b-4599a9009c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:23.199009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a6179-5a07-4e30-b84b-4599a9009c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:23.199712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a6179-5a07-4e30-b84b-4599a9009c53 None admin] RBAC: Adding query filter params (role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=ab74c2ec77cd4eedaa75ad690b7611cf, user.id=798f01254a3447e7a935dafb3ec4a5b3) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:23.200009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a6179-5a07-4e30-b84b-4599a9009c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:23.201436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a6179-5a07-4e30-b84b-4599a9009c53 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:23.206792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 555/1117] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:52:23 2022] GET /identity/v3/role_assignments?user.id=798f01254a3447e7a935dafb3ec4a5b3&scope.project.id=ab74c2ec77cd4eedaa75ad690b7611cf&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:25.158041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56663abb-1be1-47e7-a03f-206ab914cfbf None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:25.158539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56663abb-1be1-47e7-a03f-206ab914cfbf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:25.158539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56663abb-1be1-47e7-a03f-206ab914cfbf None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:25.159831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 563/1118] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:25.165052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9e3e3-aaa0-4f51-b6a1-01df854f44e4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:25.165493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9e3e3-aaa0-4f51-b6a1-01df854f44e4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:25.165595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9e3e3-aaa0-4f51-b6a1-01df854f44e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:25.183464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-88e9e3e3-aaa0-4f51-b6a1-01df854f44e4 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:25.184687 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-88e9e3e3-aaa0-4f51-b6a1-01df854f44e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:25.250516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 556/1119] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:25.272409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56663abb-1be1-47e7-a03f-206ab914cfbf None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:25.370340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56663abb-1be1-47e7-a03f-206ab914cfbf None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:25.383026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:25.402780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ef25-901d-46a3-bcb3-935806ffe960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:25.406075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ef25-901d-46a3-bcb3-935806ffe960 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:25.406075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ef25-901d-46a3-bcb3-935806ffe960 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:25.406075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ef25-901d-46a3-bcb3-935806ffe960 None admin] PATH_INFO: `/v3/roles/b2ded61baed84402a2fa33eda9462c6f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:25.406075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132ef25-901d-46a3-bcb3-935806ffe960 None admin] RBAC: Authorizing `identity:get_role(role_id=b2ded61baed84402a2fa33eda9462c6f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:25.406625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132ef25-901d-46a3-bcb3-935806ffe960 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:25.407900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 564/1120] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:25 2022] GET /identity/v3/roles/b2ded61baed84402a2fa33eda9462c6f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:25.413576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9e3e3-aaa0-4f51-b6a1-01df854f44e4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:25.476219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:25.509813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9e3e3-aaa0-4f51-b6a1-01df854f44e4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:25.547041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85a5bb-d8c6-463c-9432-f23799b75bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:25.547660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85a5bb-d8c6-463c-9432-f23799b75bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:25.547930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85a5bb-d8c6-463c-9432-f23799b75bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:25.548193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85a5bb-d8c6-463c-9432-f23799b75bd1 None admin] PATH_INFO: `/v3/users/798f01254a3447e7a935dafb3ec4a5b3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:25.549320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85a5bb-d8c6-463c-9432-f23799b75bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=798f01254a3447e7a935dafb3ec4a5b3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:25.550738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85a5bb-d8c6-463c-9432-f23799b75bd1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:25.552299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 557/1121] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:25 2022] GET /identity/v3/users/798f01254a3447e7a935dafb3ec4a5b3 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:25.558618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ef25-901d-46a3-bcb3-935806ffe960 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:25.654683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ef25-901d-46a3-bcb3-935806ffe960 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:25.687559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ee7d5-8389-4ddb-b5bc-7f0f2205ac8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:25.688100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ee7d5-8389-4ddb-b5bc-7f0f2205ac8f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:25.688371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ee7d5-8389-4ddb-b5bc-7f0f2205ac8f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:25.688664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ee7d5-8389-4ddb-b5bc-7f0f2205ac8f None admin] PATH_INFO: `/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:25.689765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ee7d5-8389-4ddb-b5bc-7f0f2205ac8f None admin] RBAC: Authorizing `identity:get_project(project_id=ab74c2ec77cd4eedaa75ad690b7611cf)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:25.691177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ee7d5-8389-4ddb-b5bc-7f0f2205ac8f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:25.692779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 565/1122] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:25 2022] GET /identity/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:25.698653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85a5bb-d8c6-463c-9432-f23799b75bd1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:25.794938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85a5bb-d8c6-463c-9432-f23799b75bd1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:25.833010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4827c-8a3c-42af-90f0-d9878990b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:25.833688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4827c-8a3c-42af-90f0-d9878990b6ce None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:25.833961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4827c-8a3c-42af-90f0-d9878990b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:25.834226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4827c-8a3c-42af-90f0-d9878990b6ce None admin] PATH_INFO: `/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf/users/798f01254a3447e7a935dafb3ec4a5b3/roles/b2ded61baed84402a2fa33eda9462c6f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:25.836363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4827c-8a3c-42af-90f0-d9878990b6ce None admin] RBAC: Authorizing `identity:create_grant(project_id=ab74c2ec77cd4eedaa75ad690b7611cf, user_id=798f01254a3447e7a935dafb3ec4a5b3, role_id=b2ded61baed84402a2fa33eda9462c6f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:25.838331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4827c-8a3c-42af-90f0-d9878990b6ce None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:25.840556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:25.966850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 558/1123] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:52:25 2022] PUT /identity/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf/users/798f01254a3447e7a935dafb3ec4a5b3/roles/b2ded61baed84402a2fa33eda9462c6f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:52:27.930072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40b8bd-e8f0-40d4-86e3-bf8681941fc6 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:27.930594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40b8bd-e8f0-40d4-86e3-bf8681941fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:27.930594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40b8bd-e8f0-40d4-86e3-bf8681941fc6 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:27.932408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 566/1124] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:27.937466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54be20a-6a71-4e58-a31a-5a0da15a90a6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:27.937858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54be20a-6a71-4e58-a31a-5a0da15a90a6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:27.938010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54be20a-6a71-4e58-a31a-5a0da15a90a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:27.956351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b54be20a-6a71-4e58-a31a-5a0da15a90a6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:27.957556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b54be20a-6a71-4e58-a31a-5a0da15a90a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:28.020629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 559/1125] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:28.042958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40b8bd-e8f0-40d4-86e3-bf8681941fc6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:28.091958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:28.141466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40b8bd-e8f0-40d4-86e3-bf8681941fc6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:28.173684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8188b2-8104-47fc-ab89-9ea50c2f2f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:28.174294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8188b2-8104-47fc-ab89-9ea50c2f2f73 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:28.174567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8188b2-8104-47fc-ab89-9ea50c2f2f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:28.174834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8188b2-8104-47fc-ab89-9ea50c2f2f73 None admin] PATH_INFO: `/v3/roles/b2ded61baed84402a2fa33eda9462c6f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:28.176207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8188b2-8104-47fc-ab89-9ea50c2f2f73 None admin] RBAC: Authorizing `identity:get_role(role_id=b2ded61baed84402a2fa33eda9462c6f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:28.177608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8188b2-8104-47fc-ab89-9ea50c2f2f73 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:28.178905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 567/1126] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:28 2022] GET /identity/v3/roles/b2ded61baed84402a2fa33eda9462c6f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:28.186117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54be20a-6a71-4e58-a31a-5a0da15a90a6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:28.280872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54be20a-6a71-4e58-a31a-5a0da15a90a6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:28.303173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:28.313894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997af7e-5796-43a3-bf81-4373226ecd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:28.318231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997af7e-5796-43a3-bf81-4373226ecd12 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:28.318231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997af7e-5796-43a3-bf81-4373226ecd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:28.318231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997af7e-5796-43a3-bf81-4373226ecd12 None admin] PATH_INFO: `/v3/users/798f01254a3447e7a935dafb3ec4a5b3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:28.318231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997af7e-5796-43a3-bf81-4373226ecd12 None admin] RBAC: Authorizing `identity:get_user(user_id=798f01254a3447e7a935dafb3ec4a5b3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:28.318231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997af7e-5796-43a3-bf81-4373226ecd12 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:28.319008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 560/1127] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:28 2022] GET /identity/v3/users/798f01254a3447e7a935dafb3ec4a5b3 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:28.324743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8188b2-8104-47fc-ab89-9ea50c2f2f73 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:28.421170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8188b2-8104-47fc-ab89-9ea50c2f2f73 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:28.437980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:28.455165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73d3c3-fde2-4613-a3c2-52e41b6ccbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:28.455751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73d3c3-fde2-4613-a3c2-52e41b6ccbff None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:28.456245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73d3c3-fde2-4613-a3c2-52e41b6ccbff None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:28.456573 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73d3c3-fde2-4613-a3c2-52e41b6ccbff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:28.521340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73d3c3-fde2-4613-a3c2-52e41b6ccbff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:28.522763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73d3c3-fde2-4613-a3c2-52e41b6ccbff None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:28.617650 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 568/1128] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:28.623613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997af7e-5796-43a3-bf81-4373226ecd12 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:28.718985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997af7e-5796-43a3-bf81-4373226ecd12 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:28.751154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedbf0d-fe76-4090-97a1-b44f13b3c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:28.751755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedbf0d-fe76-4090-97a1-b44f13b3c567 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:28.752025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedbf0d-fe76-4090-97a1-b44f13b3c567 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:28.752284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedbf0d-fe76-4090-97a1-b44f13b3c567 None admin] PATH_INFO: `/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:28.753393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedbf0d-fe76-4090-97a1-b44f13b3c567 None admin] RBAC: Authorizing `identity:get_project(project_id=ab74c2ec77cd4eedaa75ad690b7611cf)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:28.754790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedbf0d-fe76-4090-97a1-b44f13b3c567 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:28.756357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 561/1129] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:28 2022] GET /identity/v3/projects/ab74c2ec77cd4eedaa75ad690b7611cf => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:28.762469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73d3c3-fde2-4613-a3c2-52e41b6ccbff None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:28.780660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:28.859917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73d3c3-fde2-4613-a3c2-52e41b6ccbff None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:28.891482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a861089-f8db-40c0-807c-da0477045cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:28.892060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a861089-f8db-40c0-807c-da0477045cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:28.892328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a861089-f8db-40c0-807c-da0477045cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:28.892616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a861089-f8db-40c0-807c-da0477045cd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:28.893320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a861089-f8db-40c0-807c-da0477045cd8 None admin] RBAC: Adding query filter params (role.id=b2ded61baed84402a2fa33eda9462c6f, scope.project.id=ab74c2ec77cd4eedaa75ad690b7611cf, user.id=798f01254a3447e7a935dafb3ec4a5b3) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:28.893616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a861089-f8db-40c0-807c-da0477045cd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:28.895016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a861089-f8db-40c0-807c-da0477045cd8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:28.901081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 569/1130] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:52:28 2022] GET /identity/v3/role_assignments?user.id=798f01254a3447e7a935dafb3ec4a5b3&scope.project.id=ab74c2ec77cd4eedaa75ad690b7611cf&role.id=b2ded61baed84402a2fa33eda9462c6f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:30.918632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19c530-26be-4e90-9c0f-6285756b4a18 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:30.919191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19c530-26be-4e90-9c0f-6285756b4a18 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:30.919300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19c530-26be-4e90-9c0f-6285756b4a18 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:30.920623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 562/1131] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:52:30.925870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa18da-effd-48a2-bb35-ab2423f8c5b6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:30.926262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa18da-effd-48a2-bb35-ab2423f8c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:30.926401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa18da-effd-48a2-bb35-ab2423f8c5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:30.947219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4ffa18da-effd-48a2-bb35-ab2423f8c5b6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:30.948380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4ffa18da-effd-48a2-bb35-ab2423f8c5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:31.012018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 570/1132] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:31.034148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19c530-26be-4e90-9c0f-6285756b4a18 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:31.092903 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:31.132226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19c530-26be-4e90-9c0f-6285756b4a18 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:31.165588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf51d91-2113-4dfb-b8e9-553de2f6bd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:31.166185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51d91-2113-4dfb-b8e9-553de2f6bd57 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:31.166458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51d91-2113-4dfb-b8e9-553de2f6bd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:31.166725 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51d91-2113-4dfb-b8e9-553de2f6bd57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:31.167820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf51d91-2113-4dfb-b8e9-553de2f6bd57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:31.169264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf51d91-2113-4dfb-b8e9-553de2f6bd57 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:31.170786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 563/1133] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:31.176435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa18da-effd-48a2-bb35-ab2423f8c5b6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:31.188175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:31.274349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa18da-effd-48a2-bb35-ab2423f8c5b6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:31.307212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2165b0-0b28-4926-aa90-376422c1bac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:31.307798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2165b0-0b28-4926-aa90-376422c1bac8 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:31.308068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2165b0-0b28-4926-aa90-376422c1bac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:31.308331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2165b0-0b28-4926-aa90-376422c1bac8 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:31.308998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2165b0-0b28-4926-aa90-376422c1bac8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:31.310413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2165b0-0b28-4926-aa90-376422c1bac8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:31.396198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 571/1134] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:52:31 2022] POST /identity/v3/projects => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:52:33.394037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092c90b-c9ed-4bbc-8639-996e2fbdc8e8 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:33.394570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092c90b-c9ed-4bbc-8639-996e2fbdc8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:33.394570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092c90b-c9ed-4bbc-8639-996e2fbdc8e8 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:33.395937 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 564/1135] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:33.401025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22976000-248d-41e4-9f5e-adb81dea3238 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:33.401414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22976000-248d-41e4-9f5e-adb81dea3238 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:33.401561 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22976000-248d-41e4-9f5e-adb81dea3238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:33.419089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-22976000-248d-41e4-9f5e-adb81dea3238 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:33.420271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-22976000-248d-41e4-9f5e-adb81dea3238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:33.474679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:33.484909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 572/1136] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:33.508982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092c90b-c9ed-4bbc-8639-996e2fbdc8e8 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:33.609464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092c90b-c9ed-4bbc-8639-996e2fbdc8e8 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:33.637128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:33.646174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81eb2f-61f7-4043-aa44-70af4ede7209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:33.646738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81eb2f-61f7-4043-aa44-70af4ede7209 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:33.647009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81eb2f-61f7-4043-aa44-70af4ede7209 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:33.647273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81eb2f-61f7-4043-aa44-70af4ede7209 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:33.648375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81eb2f-61f7-4043-aa44-70af4ede7209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:33.649820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81eb2f-61f7-4043-aa44-70af4ede7209 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:33.651427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 565/1137] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:52:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:33.657157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22976000-248d-41e4-9f5e-adb81dea3238 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:33.753312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22976000-248d-41e4-9f5e-adb81dea3238 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:33.785509 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbb0e6-2759-49a0-ade4-0c6f2d1c9f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:33.786097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbb0e6-2759-49a0-ade4-0c6f2d1c9f42 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:33.786373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbb0e6-2759-49a0-ade4-0c6f2d1c9f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:33.786639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbb0e6-2759-49a0-ade4-0c6f2d1c9f42 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:33.787285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dbb0e6-2759-49a0-ade4-0c6f2d1c9f42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:33.788749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dbb0e6-2759-49a0-ade4-0c6f2d1c9f42 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:33.886859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 573/1138] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:52:33 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:52:35.919180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20989e3d-b44f-4a31-a656-dc0318524efc None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:35.919782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20989e3d-b44f-4a31-a656-dc0318524efc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:35.919782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20989e3d-b44f-4a31-a656-dc0318524efc None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:35.921059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 566/1139] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:35.926330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcaaeb-a2e9-43e7-a415-f91472989271 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:35.926698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcaaeb-a2e9-43e7-a415-f91472989271 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:35.926874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcaaeb-a2e9-43e7-a415-f91472989271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:35.944443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-15dcaaeb-a2e9-43e7-a415-f91472989271 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:35.945863 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-15dcaaeb-a2e9-43e7-a415-f91472989271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:35.988454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:36.010593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 574/1140] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:36.033522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20989e3d-b44f-4a31-a656-dc0318524efc None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:36.130199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20989e3d-b44f-4a31-a656-dc0318524efc None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:36.162109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409b71c-f494-4818-ac34-34cae6967549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:36.162699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409b71c-f494-4818-ac34-34cae6967549 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:36.162968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409b71c-f494-4818-ac34-34cae6967549 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:36.163229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409b71c-f494-4818-ac34-34cae6967549 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:36.172827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409b71c-f494-4818-ac34-34cae6967549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-7409b71c-f494-4818-ac34-34cae6967549 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:36.177374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:36.178566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:36.179726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 567/1141] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:52:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:36.185243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcaaeb-a2e9-43e7-a415-f91472989271 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:36.279176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcaaeb-a2e9-43e7-a415-f91472989271 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:36.311841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e747e3a-afb3-4b3f-b8a2-a6924e6fcbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:36.312424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e747e3a-afb3-4b3f-b8a2-a6924e6fcbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:36.312724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e747e3a-afb3-4b3f-b8a2-a6924e6fcbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:36.312990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e747e3a-afb3-4b3f-b8a2-a6924e6fcbd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:36.313636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e747e3a-afb3-4b3f-b8a2-a6924e6fcbd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:36.313931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e747e3a-afb3-4b3f-b8a2-a6924e6fcbd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:36.315277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e747e3a-afb3-4b3f-b8a2-a6924e6fcbd6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:36.323598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 575/1142] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:52:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:36.329592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409b71c-f494-4818-ac34-34cae6967549 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:36.383474 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:36.425584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409b71c-f494-4818-ac34-34cae6967549 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:36.460009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16342bb7-a036-4075-97e3-56f22cd2be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:36.460644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16342bb7-a036-4075-97e3-56f22cd2be4a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:36.460920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16342bb7-a036-4075-97e3-56f22cd2be4a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:36.461186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16342bb7-a036-4075-97e3-56f22cd2be4a None admin] PATH_INFO: `/v3/users/a92e4f306121432b871051bbf3943932` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:36.473726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16342bb7-a036-4075-97e3-56f22cd2be4a None admin] RBAC: Authorizing `identity:get_user(user_id=a92e4f306121432b871051bbf3943932)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:36.475156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16342bb7-a036-4075-97e3-56f22cd2be4a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:36.476742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 568/1143] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:36 2022] GET /identity/v3/users/a92e4f306121432b871051bbf3943932 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:36.482932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e747e3a-afb3-4b3f-b8a2-a6924e6fcbd6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:36.523452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:36.579941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e747e3a-afb3-4b3f-b8a2-a6924e6fcbd6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:36.612427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0c3cf-e1a7-4b70-ab6c-f3bf9e34c245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:36.614272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0c3cf-e1a7-4b70-ab6c-f3bf9e34c245 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:36.614272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0c3cf-e1a7-4b70-ab6c-f3bf9e34c245 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:36.614272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0c3cf-e1a7-4b70-ab6c-f3bf9e34c245 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:36.678092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0c3cf-e1a7-4b70-ab6c-f3bf9e34c245 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:36.679465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0c3cf-e1a7-4b70-ab6c-f3bf9e34c245 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:36.718070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:36.775075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 576/1144] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:36.781051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16342bb7-a036-4075-97e3-56f22cd2be4a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:36.877901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16342bb7-a036-4075-97e3-56f22cd2be4a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:36.891134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:36.910419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c52113-ba36-440d-a547-289081f83da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:36.911032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c52113-ba36-440d-a547-289081f83da2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:36.911434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c52113-ba36-440d-a547-289081f83da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:36.911700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c52113-ba36-440d-a547-289081f83da2 None admin] PATH_INFO: `/v3/projects/a4b0deadc77e44faaefc6b77b35128cf` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:36.912831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c52113-ba36-440d-a547-289081f83da2 None admin] RBAC: Authorizing `identity:get_project(project_id=a4b0deadc77e44faaefc6b77b35128cf)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:36.914243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c52113-ba36-440d-a547-289081f83da2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:36.916436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 569/1145] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:36 2022] GET /identity/v3/projects/a4b0deadc77e44faaefc6b77b35128cf => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:36.922656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0c3cf-e1a7-4b70-ab6c-f3bf9e34c245 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:37.021992 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0c3cf-e1a7-4b70-ab6c-f3bf9e34c245 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:37.055756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f257e-bd74-4a2d-8db6-2903e0d60988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:37.056337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f257e-bd74-4a2d-8db6-2903e0d60988 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:37.056639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f257e-bd74-4a2d-8db6-2903e0d60988 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:37.056904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f257e-bd74-4a2d-8db6-2903e0d60988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:37.057616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f257e-bd74-4a2d-8db6-2903e0d60988 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=a4b0deadc77e44faaefc6b77b35128cf, user.id=a92e4f306121432b871051bbf3943932) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:37.057915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f257e-bd74-4a2d-8db6-2903e0d60988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:37.059335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f257e-bd74-4a2d-8db6-2903e0d60988 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:37.064445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 577/1146] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:52:36 2022] GET /identity/v3/role_assignments?user.id=a92e4f306121432b871051bbf3943932&scope.project.id=a4b0deadc77e44faaefc6b77b35128cf&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:39.021953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c3244-d25c-45f5-8308-707cdb3bb212 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:39.022473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c3244-d25c-45f5-8308-707cdb3bb212 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:39.022584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c3244-d25c-45f5-8308-707cdb3bb212 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:39.023817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 570/1147] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:39.029473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5357e3-b4a2-4f28-b92f-4e702b756cdc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:39.029854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5357e3-b4a2-4f28-b92f-4e702b756cdc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:39.030012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5357e3-b4a2-4f28-b92f-4e702b756cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:39.037094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:39.047363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7e5357e3-b4a2-4f28-b92f-4e702b756cdc None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:39.049980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7e5357e3-b4a2-4f28-b92f-4e702b756cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:39.113294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 578/1148] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:52:39.136028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c3244-d25c-45f5-8308-707cdb3bb212 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:39.240958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c3244-d25c-45f5-8308-707cdb3bb212 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:39.274333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698446a-fca8-496f-8a4f-f635b9629dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:39.274934 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698446a-fca8-496f-8a4f-f635b9629dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:39.275202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698446a-fca8-496f-8a4f-f635b9629dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:39.275461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698446a-fca8-496f-8a4f-f635b9629dd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:39.284943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698446a-fca8-496f-8a4f-f635b9629dd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-c698446a-fca8-496f-8a4f-f635b9629dd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:39.289546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:39.290635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:39.291696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 571/1149] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:52:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:39.296731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5357e3-b4a2-4f28-b92f-4e702b756cdc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:39.394152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5357e3-b4a2-4f28-b92f-4e702b756cdc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:39.425993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c02c7-9e2a-447f-840f-e0aebbe8d660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:39.426573 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c02c7-9e2a-447f-840f-e0aebbe8d660 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:39.426848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c02c7-9e2a-447f-840f-e0aebbe8d660 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:39.427113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c02c7-9e2a-447f-840f-e0aebbe8d660 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:39.427764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c02c7-9e2a-447f-840f-e0aebbe8d660 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:39.428059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c02c7-9e2a-447f-840f-e0aebbe8d660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:39.429440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c02c7-9e2a-447f-840f-e0aebbe8d660 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:39.431258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:39.438183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 579/1150] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:52:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:39.444174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698446a-fca8-496f-8a4f-f635b9629dd5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:39.486805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:39.541731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698446a-fca8-496f-8a4f-f635b9629dd5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:39.574349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9521c13-ba0b-439e-8163-5e87974aa032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:39.574951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9521c13-ba0b-439e-8163-5e87974aa032 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:39.575224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9521c13-ba0b-439e-8163-5e87974aa032 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:39.575492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9521c13-ba0b-439e-8163-5e87974aa032 None admin] PATH_INFO: `/v3/users/a92e4f306121432b871051bbf3943932` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:39.576619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9521c13-ba0b-439e-8163-5e87974aa032 None admin] RBAC: Authorizing `identity:get_user(user_id=a92e4f306121432b871051bbf3943932)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:39.581843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9521c13-ba0b-439e-8163-5e87974aa032 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:39.581843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 572/1151] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:39 2022] GET /identity/v3/users/a92e4f306121432b871051bbf3943932 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:39.585490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c02c7-9e2a-447f-840f-e0aebbe8d660 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:39.681246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c02c7-9e2a-447f-840f-e0aebbe8d660 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:39.712751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080ce95-52e8-435e-b0dc-1683f84b1790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:39.713354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080ce95-52e8-435e-b0dc-1683f84b1790 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:39.713626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080ce95-52e8-435e-b0dc-1683f84b1790 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:39.713894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080ce95-52e8-435e-b0dc-1683f84b1790 None admin] PATH_INFO: `/v3/projects/a4b0deadc77e44faaefc6b77b35128cf` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:39.714993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080ce95-52e8-435e-b0dc-1683f84b1790 None admin] RBAC: Authorizing `identity:get_project(project_id=a4b0deadc77e44faaefc6b77b35128cf)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:39.716420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080ce95-52e8-435e-b0dc-1683f84b1790 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:39.718041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 580/1152] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:39 2022] GET /identity/v3/projects/a4b0deadc77e44faaefc6b77b35128cf => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:39.723718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9521c13-ba0b-439e-8163-5e87974aa032 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:39.820368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9521c13-ba0b-439e-8163-5e87974aa032 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:39.830957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:39.854291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9db91-e42d-4200-b9e8-4d917af7a805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:39.854930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9db91-e42d-4200-b9e8-4d917af7a805 None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:39.855205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9db91-e42d-4200-b9e8-4d917af7a805 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:39.862134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9db91-e42d-4200-b9e8-4d917af7a805 None admin] PATH_INFO: `/v3/projects/a4b0deadc77e44faaefc6b77b35128cf/users/a92e4f306121432b871051bbf3943932/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:39.862134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9db91-e42d-4200-b9e8-4d917af7a805 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b0deadc77e44faaefc6b77b35128cf, user_id=a92e4f306121432b871051bbf3943932, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:39.862134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9db91-e42d-4200-b9e8-4d917af7a805 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:39.943388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 573/1153] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:52:39 2022] PUT /identity/v3/projects/a4b0deadc77e44faaefc6b77b35128cf/users/a92e4f306121432b871051bbf3943932/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:52:41.911297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412d8cf-4813-404f-975e-6244184560b0 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:41.911826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412d8cf-4813-404f-975e-6244184560b0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:41.911826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412d8cf-4813-404f-975e-6244184560b0 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:41.913189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 581/1154] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:41.918866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e83b0-be08-4d9b-ae30-daaa74ea3b17 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:41.919271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e83b0-be08-4d9b-ae30-daaa74ea3b17 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:41.919412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e83b0-be08-4d9b-ae30-daaa74ea3b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:41.937597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-984e83b0-be08-4d9b-ae30-daaa74ea3b17 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:41.938779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-984e83b0-be08-4d9b-ae30-daaa74ea3b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:42.004808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 574/1155] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:42.028265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412d8cf-4813-404f-975e-6244184560b0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:42.089767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:42.126166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412d8cf-4813-404f-975e-6244184560b0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:42.157919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d88a-d558-4b33-8af8-a99da73e1805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:42.158518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d88a-d558-4b33-8af8-a99da73e1805 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:42.158789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d88a-d558-4b33-8af8-a99da73e1805 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:42.159053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d88a-d558-4b33-8af8-a99da73e1805 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:42.169690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8d88a-d558-4b33-8af8-a99da73e1805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d4d8d88a-d558-4b33-8af8-a99da73e1805 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:42.175501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:42.176641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:42.177685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 582/1156] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:52:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:42.182496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e83b0-be08-4d9b-ae30-daaa74ea3b17 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:42.274498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:42.277306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e83b0-be08-4d9b-ae30-daaa74ea3b17 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:42.310312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f213a-96ee-4d1e-85b8-da4de2501a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:42.310902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f213a-96ee-4d1e-85b8-da4de2501a04 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:42.311170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f213a-96ee-4d1e-85b8-da4de2501a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:42.311432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f213a-96ee-4d1e-85b8-da4de2501a04 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:42.312078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f213a-96ee-4d1e-85b8-da4de2501a04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:42.312370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f213a-96ee-4d1e-85b8-da4de2501a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:42.313740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f213a-96ee-4d1e-85b8-da4de2501a04 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:42.321363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 575/1157] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:52:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:42.326901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d88a-d558-4b33-8af8-a99da73e1805 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:42.423599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d88a-d558-4b33-8af8-a99da73e1805 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:42.445331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:42.456269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6be839-e72c-4dab-99b5-d2879fb43a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:42.456913 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6be839-e72c-4dab-99b5-d2879fb43a0a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:42.457185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6be839-e72c-4dab-99b5-d2879fb43a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:42.457450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6be839-e72c-4dab-99b5-d2879fb43a0a None admin] PATH_INFO: `/v3/users/a92e4f306121432b871051bbf3943932` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:42.458548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6be839-e72c-4dab-99b5-d2879fb43a0a None admin] RBAC: Authorizing `identity:get_user(user_id=a92e4f306121432b871051bbf3943932)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:42.459965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6be839-e72c-4dab-99b5-d2879fb43a0a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:42.462143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 583/1158] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:42 2022] GET /identity/v3/users/a92e4f306121432b871051bbf3943932 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:42.467956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f213a-96ee-4d1e-85b8-da4de2501a04 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:42.562211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f213a-96ee-4d1e-85b8-da4de2501a04 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:42.595068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240ad3-8f3e-42e6-bbb2-e02445a516f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:42.595639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240ad3-8f3e-42e6-bbb2-e02445a516f1 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:42.595907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240ad3-8f3e-42e6-bbb2-e02445a516f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:42.596167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240ad3-8f3e-42e6-bbb2-e02445a516f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:42.624387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:42.660616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab240ad3-8f3e-42e6-bbb2-e02445a516f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:42.662029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab240ad3-8f3e-42e6-bbb2-e02445a516f1 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:42.754694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 576/1159] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:42.761021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6be839-e72c-4dab-99b5-d2879fb43a0a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:42.858048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6be839-e72c-4dab-99b5-d2879fb43a0a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:42.891754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcd72d-6771-443e-b89e-a562ec48fbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:42.892360 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcd72d-6771-443e-b89e-a562ec48fbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:42.892665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcd72d-6771-443e-b89e-a562ec48fbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:42.892933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcd72d-6771-443e-b89e-a562ec48fbf3 None admin] PATH_INFO: `/v3/projects/a4b0deadc77e44faaefc6b77b35128cf` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:42.894040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcd72d-6771-443e-b89e-a562ec48fbf3 None admin] RBAC: Authorizing `identity:get_project(project_id=a4b0deadc77e44faaefc6b77b35128cf)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:42.895463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcd72d-6771-443e-b89e-a562ec48fbf3 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:42.897100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 584/1160] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:42 2022] GET /identity/v3/projects/a4b0deadc77e44faaefc6b77b35128cf => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:42.903135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240ad3-8f3e-42e6-bbb2-e02445a516f1 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:42.961369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:42.999979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240ad3-8f3e-42e6-bbb2-e02445a516f1 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:43.033596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad29e51-af78-4af6-891e-569e85cbe80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:43.034120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad29e51-af78-4af6-891e-569e85cbe80d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:43.034390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad29e51-af78-4af6-891e-569e85cbe80d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:43.034655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad29e51-af78-4af6-891e-569e85cbe80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:43.035366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad29e51-af78-4af6-891e-569e85cbe80d None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=a4b0deadc77e44faaefc6b77b35128cf, user.id=a92e4f306121432b871051bbf3943932) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:43.035661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad29e51-af78-4af6-891e-569e85cbe80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:43.037095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad29e51-af78-4af6-891e-569e85cbe80d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:43.042583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 577/1161] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:52:42 2022] GET /identity/v3/role_assignments?user.id=a92e4f306121432b871051bbf3943932&scope.project.id=a4b0deadc77e44faaefc6b77b35128cf&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:45.046567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecf88a-5d47-478d-9661-8687ad1be509 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:45.047084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecf88a-5d47-478d-9661-8687ad1be509 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:45.047084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecf88a-5d47-478d-9661-8687ad1be509 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:45.048412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 585/1162] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:45.053617 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90391-5074-4cd2-9667-d6b368b2b483 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:45.054032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90391-5074-4cd2-9667-d6b368b2b483 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:45.054135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90391-5074-4cd2-9667-d6b368b2b483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:45.072162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-44d90391-5074-4cd2-9667-d6b368b2b483 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:45.073362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-44d90391-5074-4cd2-9667-d6b368b2b483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:45.137135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 578/1163] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:45.158886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecf88a-5d47-478d-9661-8687ad1be509 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:45.217595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:45.260701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecf88a-5d47-478d-9661-8687ad1be509 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:45.297679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c84577-d482-4292-9aea-079ee09dc977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:45.298262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c84577-d482-4292-9aea-079ee09dc977 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:45.298533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c84577-d482-4292-9aea-079ee09dc977 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:45.298794 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c84577-d482-4292-9aea-079ee09dc977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:45.362492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c84577-d482-4292-9aea-079ee09dc977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:45.363862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c84577-d482-4292-9aea-079ee09dc977 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:45.419588 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:45.459408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 586/1164] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:45.465462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d90391-5074-4cd2-9667-d6b368b2b483 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:45.519100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:45.560688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d90391-5074-4cd2-9667-d6b368b2b483 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:45.593666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433aef61-27de-4c8d-b447-c3962bda9716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:45.594268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aef61-27de-4c8d-b447-c3962bda9716 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:45.594533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aef61-27de-4c8d-b447-c3962bda9716 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:45.594792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aef61-27de-4c8d-b447-c3962bda9716 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:45.601114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433aef61-27de-4c8d-b447-c3962bda9716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:45.603023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433aef61-27de-4c8d-b447-c3962bda9716 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-433aef61-27de-4c8d-b447-c3962bda9716 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:45.616884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:45.618756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:52:45.619848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 579/1165] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:52:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:45.621663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c84577-d482-4292-9aea-079ee09dc977 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:45.715166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c84577-d482-4292-9aea-079ee09dc977 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:45.747802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f366e8-7341-4696-a748-e34ced1c3c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:45.748380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f366e8-7341-4696-a748-e34ced1c3c3c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:45.748677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f366e8-7341-4696-a748-e34ced1c3c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:45.748937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f366e8-7341-4696-a748-e34ced1c3c3c None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:45.749555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f366e8-7341-4696-a748-e34ced1c3c3c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:45.749844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f366e8-7341-4696-a748-e34ced1c3c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:45.751181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f366e8-7341-4696-a748-e34ced1c3c3c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:45.758068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 587/1166] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 17:52:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:47.736259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846fce-a4db-44b8-9e48-81e422cbd592 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:47.736837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846fce-a4db-44b8-9e48-81e422cbd592 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:47.736837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846fce-a4db-44b8-9e48-81e422cbd592 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:47.738139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 580/1167] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:47.743414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea0a0b-4784-4ec8-87c9-ce8d6dfe1a9e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:47.743854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea0a0b-4784-4ec8-87c9-ce8d6dfe1a9e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:47.743949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea0a0b-4784-4ec8-87c9-ce8d6dfe1a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:47.762103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7cea0a0b-4784-4ec8-87c9-ce8d6dfe1a9e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:47.763300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7cea0a0b-4784-4ec8-87c9-ce8d6dfe1a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:47.780781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:47.829539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 588/1168] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:47.852853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846fce-a4db-44b8-9e48-81e422cbd592 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:47.951009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846fce-a4db-44b8-9e48-81e422cbd592 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:47.967367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:47.983960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fedb81-e024-47cd-9133-7e1e7b2af043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:47.984566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fedb81-e024-47cd-9133-7e1e7b2af043 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:47.984841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fedb81-e024-47cd-9133-7e1e7b2af043 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:47.985104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fedb81-e024-47cd-9133-7e1e7b2af043 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:47.985754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fedb81-e024-47cd-9133-7e1e7b2af043 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:47.987150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fedb81-e024-47cd-9133-7e1e7b2af043 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:48.058601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 581/1169] 10.222.0.50 () {60 vars in 1138 bytes} [Mon Aug 8 17:52:47 2022] POST /identity/v3/domains => generated 267 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:52:50.075106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af4f9e-d9f8-4948-a409-e12f2c3d038d None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:50.075621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af4f9e-d9f8-4948-a409-e12f2c3d038d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:50.075621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af4f9e-d9f8-4948-a409-e12f2c3d038d None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:50.076918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 589/1170] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:50.082208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa75aa-a883-4d5d-8df3-1195adebf20b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:50.082576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa75aa-a883-4d5d-8df3-1195adebf20b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:50.082754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa75aa-a883-4d5d-8df3-1195adebf20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:50.100207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-34fa75aa-a883-4d5d-8df3-1195adebf20b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:50.101939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-34fa75aa-a883-4d5d-8df3-1195adebf20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:50.165515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 582/1171] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:50.188645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af4f9e-d9f8-4948-a409-e12f2c3d038d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:50.286015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af4f9e-d9f8-4948-a409-e12f2c3d038d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:50.319934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc53c34-3559-4dcc-abe4-fc12a0f8feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:50.320557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc53c34-3559-4dcc-abe4-fc12a0f8feb0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:50.320832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc53c34-3559-4dcc-abe4-fc12a0f8feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:50.321093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc53c34-3559-4dcc-abe4-fc12a0f8feb0 None admin] PATH_INFO: `/v3/domains/4b6dd051ab28477480d399134d7e1a21` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:50.331481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc53c34-3559-4dcc-abe4-fc12a0f8feb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4b6dd051ab28477480d399134d7e1a21)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:50.332885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc53c34-3559-4dcc-abe4-fc12a0f8feb0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:50.334459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 590/1172] 10.222.0.50 () {56 vars in 1183 bytes} [Mon Aug 8 17:52:50 2022] GET /identity/v3/domains/4b6dd051ab28477480d399134d7e1a21 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:50.340704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa75aa-a883-4d5d-8df3-1195adebf20b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:50.434860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa75aa-a883-4d5d-8df3-1195adebf20b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:50.441746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:50.467091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c45bd-640c-4150-870a-a3e0df905c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:50.469147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c45bd-640c-4150-870a-a3e0df905c78 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:50.469147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c45bd-640c-4150-870a-a3e0df905c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:50.469147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c45bd-640c-4150-870a-a3e0df905c78 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:50.469147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c45bd-640c-4150-870a-a3e0df905c78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:50.470297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c45bd-640c-4150-870a-a3e0df905c78 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:50.584792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 583/1173] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:52:50 2022] POST /identity/v3/projects => generated 361 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:52:52.590545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5ed1f-7ecd-44e6-a134-b050236d7070 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:52.591102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5ed1f-7ecd-44e6-a134-b050236d7070 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:52.591102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5ed1f-7ecd-44e6-a134-b050236d7070 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:52.592412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 591/1174] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:52.597444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e300bb7-7074-446f-b0e7-0798a4d53271 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:52.597834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e300bb7-7074-446f-b0e7-0798a4d53271 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:52.597980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e300bb7-7074-446f-b0e7-0798a4d53271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:52.615260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2e300bb7-7074-446f-b0e7-0798a4d53271 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:52.616422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2e300bb7-7074-446f-b0e7-0798a4d53271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:52.680257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 584/1175] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:52.704916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5ed1f-7ecd-44e6-a134-b050236d7070 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:52.720683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:52.804063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5ed1f-7ecd-44e6-a134-b050236d7070 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:52.837366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f454a65-c888-4351-bc84-6a6c03137f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:52.837984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f454a65-c888-4351-bc84-6a6c03137f96 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:52.838262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f454a65-c888-4351-bc84-6a6c03137f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:52.838530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f454a65-c888-4351-bc84-6a6c03137f96 None admin] PATH_INFO: `/v3/domains/4b6dd051ab28477480d399134d7e1a21` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:52.839630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f454a65-c888-4351-bc84-6a6c03137f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4b6dd051ab28477480d399134d7e1a21)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:52.841137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f454a65-c888-4351-bc84-6a6c03137f96 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:52.842655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 592/1176] 10.222.0.50 () {56 vars in 1183 bytes} [Mon Aug 8 17:52:52 2022] GET /identity/v3/domains/4b6dd051ab28477480d399134d7e1a21 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:52.848350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e300bb7-7074-446f-b0e7-0798a4d53271 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:52.932295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:52.943587 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e300bb7-7074-446f-b0e7-0798a4d53271 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:52.978214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e95c4-fb48-4876-a8c2-92350d6e301d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:52.978795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e95c4-fb48-4876-a8c2-92350d6e301d None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:52.979067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e95c4-fb48-4876-a8c2-92350d6e301d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:52.979330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e95c4-fb48-4876-a8c2-92350d6e301d None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:52.979950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e95c4-fb48-4876-a8c2-92350d6e301d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:52.981388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e95c4-fb48-4876-a8c2-92350d6e301d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:53.066660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 585/1177] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:52:52 2022] POST /identity/v3/users => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:52:55.089776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77683e28-1708-41e9-b424-6115549a9920 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:55.090274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77683e28-1708-41e9-b424-6115549a9920 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:55.090274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77683e28-1708-41e9-b424-6115549a9920 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:55.091556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 593/1178] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:55.096766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1e50a-0968-4eda-9117-45224a460050 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:55.097207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1e50a-0968-4eda-9117-45224a460050 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:55.097307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1e50a-0968-4eda-9117-45224a460050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:55.115461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c1d1e50a-0968-4eda-9117-45224a460050 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:55.116669 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c1d1e50a-0968-4eda-9117-45224a460050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:55.180418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 586/1179] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:55.203520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77683e28-1708-41e9-b424-6115549a9920 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:55.284382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:55.302304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77683e28-1708-41e9-b424-6115549a9920 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:55.335103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d9798-2305-4769-97af-a875c4e5fe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:55.336302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d9798-2305-4769-97af-a875c4e5fe2b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:55.336602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d9798-2305-4769-97af-a875c4e5fe2b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:55.336866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d9798-2305-4769-97af-a875c4e5fe2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:55.346564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d9798-2305-4769-97af-a875c4e5fe2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-df0d9798-2305-4769-97af-a875c4e5fe2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:55.351234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:55.352363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:55.353440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 594/1180] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:52:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:55.358280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1e50a-0968-4eda-9117-45224a460050 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:55.428793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:55.452289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1e50a-0968-4eda-9117-45224a460050 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:55.485452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb743a-4407-4458-b13e-9af8bb769739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:55.486033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb743a-4407-4458-b13e-9af8bb769739 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:55.486306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb743a-4407-4458-b13e-9af8bb769739 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:55.486568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb743a-4407-4458-b13e-9af8bb769739 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:55.487215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb743a-4407-4458-b13e-9af8bb769739 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:55.487510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb743a-4407-4458-b13e-9af8bb769739 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:55.488880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb743a-4407-4458-b13e-9af8bb769739 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:55.496488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 587/1181] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:52:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:55.502126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d9798-2305-4769-97af-a875c4e5fe2b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:55.599064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d9798-2305-4769-97af-a875c4e5fe2b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:55.631015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cdfad-ab1c-43e7-aa0b-06e06694091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:55.631616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cdfad-ab1c-43e7-aa0b-06e06694091f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:55.631889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cdfad-ab1c-43e7-aa0b-06e06694091f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:55.632155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cdfad-ab1c-43e7-aa0b-06e06694091f None admin] PATH_INFO: `/v3/users/666cfdf9ef714f5faa84cb34910c2d23` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:55.643387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:55.645744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cdfad-ab1c-43e7-aa0b-06e06694091f None admin] RBAC: Authorizing `identity:get_user(user_id=666cfdf9ef714f5faa84cb34910c2d23)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:55.647235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cdfad-ab1c-43e7-aa0b-06e06694091f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:55.652422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 595/1182] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:55 2022] GET /identity/v3/users/666cfdf9ef714f5faa84cb34910c2d23 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:55.654670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb743a-4407-4458-b13e-9af8bb769739 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:55.754212 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb743a-4407-4458-b13e-9af8bb769739 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:55.786272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcf545-d262-4ccf-b959-1146ddc940ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:55.786882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf545-d262-4ccf-b959-1146ddc940ef None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:55.787153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf545-d262-4ccf-b959-1146ddc940ef None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:55.787418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf545-d262-4ccf-b959-1146ddc940ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:55.790795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:55.853942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcf545-d262-4ccf-b959-1146ddc940ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:55.855330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcf545-d262-4ccf-b959-1146ddc940ef None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:55.949804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 588/1183] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:52:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:52:55.955966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cdfad-ab1c-43e7-aa0b-06e06694091f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:56.052200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cdfad-ab1c-43e7-aa0b-06e06694091f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:56.085049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca8721-891c-4e22-875c-ab9857d9102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:56.085624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca8721-891c-4e22-875c-ab9857d9102b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:56.085890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca8721-891c-4e22-875c-ab9857d9102b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:56.086152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca8721-891c-4e22-875c-ab9857d9102b None admin] PATH_INFO: `/v3/projects/96d5a3181e4546859e0dd7835abfd4b5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:56.087233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca8721-891c-4e22-875c-ab9857d9102b None admin] RBAC: Authorizing `identity:get_project(project_id=96d5a3181e4546859e0dd7835abfd4b5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:56.088668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca8721-891c-4e22-875c-ab9857d9102b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:56.090234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 596/1184] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:55 2022] GET /identity/v3/projects/96d5a3181e4546859e0dd7835abfd4b5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:56.096204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcf545-d262-4ccf-b959-1146ddc940ef None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:56.130028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:56.191726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcf545-d262-4ccf-b959-1146ddc940ef None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:56.224827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda86fdb-b76f-4199-a4dd-89226989e427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:56.225402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86fdb-b76f-4199-a4dd-89226989e427 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:56.225668 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86fdb-b76f-4199-a4dd-89226989e427 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:56.225924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86fdb-b76f-4199-a4dd-89226989e427 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:56.227209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda86fdb-b76f-4199-a4dd-89226989e427 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=96d5a3181e4546859e0dd7835abfd4b5, user.id=666cfdf9ef714f5faa84cb34910c2d23) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:56.227502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda86fdb-b76f-4199-a4dd-89226989e427 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:56.228918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda86fdb-b76f-4199-a4dd-89226989e427 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:56.233964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 589/1185] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:52:56 2022] GET /identity/v3/role_assignments?user.id=666cfdf9ef714f5faa84cb34910c2d23&scope.project.id=96d5a3181e4546859e0dd7835abfd4b5&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:58.200891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458aa5c-2a46-41a6-8f4e-20da5a892f7d None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:58.201403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458aa5c-2a46-41a6-8f4e-20da5a892f7d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:58.201403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458aa5c-2a46-41a6-8f4e-20da5a892f7d None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:58.202926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 597/1186] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:52:58.208224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6dcba-fc40-4c4a-b3e5-e11779e660f3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:58.208672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6dcba-fc40-4c4a-b3e5-e11779e660f3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:58.208806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6dcba-fc40-4c4a-b3e5-e11779e660f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:58.226927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4ce6dcba-fc40-4c4a-b3e5-e11779e660f3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:52:58.228102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4ce6dcba-fc40-4c4a-b3e5-e11779e660f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:52:58.291884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 590/1187] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:52:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:52:58.313224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458aa5c-2a46-41a6-8f4e-20da5a892f7d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:58.378972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:58.411547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458aa5c-2a46-41a6-8f4e-20da5a892f7d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:58.445571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589b1f0-6a73-43e4-b5f7-52c70709c8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:58.446169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589b1f0-6a73-43e4-b5f7-52c70709c8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:58.446439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589b1f0-6a73-43e4-b5f7-52c70709c8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:58.446701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589b1f0-6a73-43e4-b5f7-52c70709c8b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:58.456657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589b1f0-6a73-43e4-b5f7-52c70709c8b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-9589b1f0-6a73-43e4-b5f7-52c70709c8b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:52:58.461743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:52:58.462842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:52:58.463907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 598/1188] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:52:58.468828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6dcba-fc40-4c4a-b3e5-e11779e660f3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:58.496727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:58.561823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6dcba-fc40-4c4a-b3e5-e11779e660f3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:58.594666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb77f8-ec90-42a2-9553-990ecb3be1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:58.595242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb77f8-ec90-42a2-9553-990ecb3be1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:58.595509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb77f8-ec90-42a2-9553-990ecb3be1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:58.595768 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb77f8-ec90-42a2-9553-990ecb3be1c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:58.596416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb77f8-ec90-42a2-9553-990ecb3be1c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:52:58.596731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb77f8-ec90-42a2-9553-990ecb3be1c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:58.598072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb77f8-ec90-42a2-9553-990ecb3be1c2 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:58.605661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 591/1189] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:58.611355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589b1f0-6a73-43e4-b5f7-52c70709c8b5 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:58.707998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589b1f0-6a73-43e4-b5f7-52c70709c8b5 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:58.731995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:58.739295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89de483-4359-4574-b7cf-991332a1a578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:58.739892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89de483-4359-4574-b7cf-991332a1a578 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:58.740191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89de483-4359-4574-b7cf-991332a1a578 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:58.740448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89de483-4359-4574-b7cf-991332a1a578 None admin] PATH_INFO: `/v3/users/666cfdf9ef714f5faa84cb34910c2d23` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:58.742073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89de483-4359-4574-b7cf-991332a1a578 None admin] RBAC: Authorizing `identity:get_user(user_id=666cfdf9ef714f5faa84cb34910c2d23)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:58.743481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89de483-4359-4574-b7cf-991332a1a578 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:58.745046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 599/1190] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:52:58 2022] GET /identity/v3/users/666cfdf9ef714f5faa84cb34910c2d23 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:58.750741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb77f8-ec90-42a2-9553-990ecb3be1c2 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:58.841735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:52:58.850178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb77f8-ec90-42a2-9553-990ecb3be1c2 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:58.883208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b885adf-ce2a-4c5e-89c4-ce082feebe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:58.883827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b885adf-ce2a-4c5e-89c4-ce082feebe5b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:58.884103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b885adf-ce2a-4c5e-89c4-ce082feebe5b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:58.884369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b885adf-ce2a-4c5e-89c4-ce082feebe5b None admin] PATH_INFO: `/v3/projects/96d5a3181e4546859e0dd7835abfd4b5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:58.885512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b885adf-ce2a-4c5e-89c4-ce082feebe5b None admin] RBAC: Authorizing `identity:get_project(project_id=96d5a3181e4546859e0dd7835abfd4b5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:58.886930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b885adf-ce2a-4c5e-89c4-ce082feebe5b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:58.888554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 592/1191] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:52:58 2022] GET /identity/v3/projects/96d5a3181e4546859e0dd7835abfd4b5 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:52:58.894475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89de483-4359-4574-b7cf-991332a1a578 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:52:58.991424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89de483-4359-4574-b7cf-991332a1a578 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:52:59.026140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c85fa-eb15-4801-bd09-406856c9cb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:52:59.026793 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c85fa-eb15-4801-bd09-406856c9cb5f None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:52:59.027069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c85fa-eb15-4801-bd09-406856c9cb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:52:59.027433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c85fa-eb15-4801-bd09-406856c9cb5f None admin] PATH_INFO: `/v3/projects/96d5a3181e4546859e0dd7835abfd4b5/users/666cfdf9ef714f5faa84cb34910c2d23/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:52:59.029246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c85fa-eb15-4801-bd09-406856c9cb5f None admin] RBAC: Authorizing `identity:create_grant(project_id=96d5a3181e4546859e0dd7835abfd4b5, user_id=666cfdf9ef714f5faa84cb34910c2d23, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:52:59.030719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c85fa-eb15-4801-bd09-406856c9cb5f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:52:59.102548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 600/1192] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:52:58 2022] PUT /identity/v3/projects/96d5a3181e4546859e0dd7835abfd4b5/users/666cfdf9ef714f5faa84cb34910c2d23/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:53:01.078808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9b6a5-18a5-41ec-bbfc-01e012aa2123 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:53:01.079306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9b6a5-18a5-41ec-bbfc-01e012aa2123 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:53:01.079306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9b6a5-18a5-41ec-bbfc-01e012aa2123 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:53:01.080602 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 593/1193] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:53:01.085828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec50613-dc4a-420b-8b51-5356259b6f8b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:53:01.086268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec50613-dc4a-420b-8b51-5356259b6f8b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:53:01.086373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec50613-dc4a-420b-8b51-5356259b6f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:53:01.103936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-bec50613-dc4a-420b-8b51-5356259b6f8b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:53:01.105138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bec50613-dc4a-420b-8b51-5356259b6f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:53:01.153978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:53:01.169301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 601/1194] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:53:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:53:01.192394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9b6a5-18a5-41ec-bbfc-01e012aa2123 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:53:01.291749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9b6a5-18a5-41ec-bbfc-01e012aa2123 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:53:01.324268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3db8-7cec-4280-923f-9612461067c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:53:01.324901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3db8-7cec-4280-923f-9612461067c7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:53:01.325177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3db8-7cec-4280-923f-9612461067c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:53:01.325444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3db8-7cec-4280-923f-9612461067c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:53:01.336054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd3db8-7cec-4280-923f-9612461067c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-91bd3db8-7cec-4280-923f-9612461067c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:53:01.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:53:01.341772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:53:01.342962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 594/1195] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:53:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:53:01.347760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec50613-dc4a-420b-8b51-5356259b6f8b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:53:01.443390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec50613-dc4a-420b-8b51-5356259b6f8b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:53:01.476298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c29754-6c6d-4f37-9f2f-13db46c68397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:53:01.476915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c29754-6c6d-4f37-9f2f-13db46c68397 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:53:01.477195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c29754-6c6d-4f37-9f2f-13db46c68397 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:53:01.477459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c29754-6c6d-4f37-9f2f-13db46c68397 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:53:01.478108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c29754-6c6d-4f37-9f2f-13db46c68397 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:53:01.478404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c29754-6c6d-4f37-9f2f-13db46c68397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:53:01.479757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c29754-6c6d-4f37-9f2f-13db46c68397 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:53:01.487455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 602/1196] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:53:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:53:01.493543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3db8-7cec-4280-923f-9612461067c7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:53:01.514753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:53:01.589613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3db8-7cec-4280-923f-9612461067c7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:53:01.621621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589fae3-a8c2-4745-86eb-59e2fa9b1858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:53:01.622220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589fae3-a8c2-4745-86eb-59e2fa9b1858 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:53:01.622493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589fae3-a8c2-4745-86eb-59e2fa9b1858 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:53:01.622760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589fae3-a8c2-4745-86eb-59e2fa9b1858 None admin] PATH_INFO: `/v3/users/666cfdf9ef714f5faa84cb34910c2d23` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:53:01.623853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589fae3-a8c2-4745-86eb-59e2fa9b1858 None admin] RBAC: Authorizing `identity:get_user(user_id=666cfdf9ef714f5faa84cb34910c2d23)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:53:01.625292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589fae3-a8c2-4745-86eb-59e2fa9b1858 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:53:01.626834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 595/1197] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 17:53:01 2022] GET /identity/v3/users/666cfdf9ef714f5faa84cb34910c2d23 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:53:01.632560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c29754-6c6d-4f37-9f2f-13db46c68397 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:53:01.682160 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:53:01.732684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c29754-6c6d-4f37-9f2f-13db46c68397 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:53:01.764485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b3ad9-1ba1-4d0f-932f-c6bf3dae792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:53:01.765137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b3ad9-1ba1-4d0f-932f-c6bf3dae792a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:53:01.765411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b3ad9-1ba1-4d0f-932f-c6bf3dae792a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:53:01.765678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b3ad9-1ba1-4d0f-932f-c6bf3dae792a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:53:01.830939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b3ad9-1ba1-4d0f-932f-c6bf3dae792a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:53:01.832373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b3ad9-1ba1-4d0f-932f-c6bf3dae792a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:53:01.878607 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:53:01.927824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 603/1198] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:53:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:53:01.933817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589fae3-a8c2-4745-86eb-59e2fa9b1858 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:53:02.017730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:53:02.029492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589fae3-a8c2-4745-86eb-59e2fa9b1858 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:53:02.060994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d8409-2e95-407d-bf6f-0c72f764043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:53:02.061588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d8409-2e95-407d-bf6f-0c72f764043b None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:53:02.061854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d8409-2e95-407d-bf6f-0c72f764043b None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:53:02.062110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d8409-2e95-407d-bf6f-0c72f764043b None admin] PATH_INFO: `/v3/projects/96d5a3181e4546859e0dd7835abfd4b5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:53:02.063187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d8409-2e95-407d-bf6f-0c72f764043b None admin] RBAC: Authorizing `identity:get_project(project_id=96d5a3181e4546859e0dd7835abfd4b5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:53:02.064616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d8409-2e95-407d-bf6f-0c72f764043b None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:53:02.066185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 596/1199] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 17:53:01 2022] GET /identity/v3/projects/96d5a3181e4546859e0dd7835abfd4b5 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:53:02.072687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b3ad9-1ba1-4d0f-932f-c6bf3dae792a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:53:02.166980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b3ad9-1ba1-4d0f-932f-c6bf3dae792a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:53:02.199194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded8cc9-74d4-49d8-a0c8-d341f308ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:53:02.199766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded8cc9-74d4-49d8-a0c8-d341f308ba13 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:53:02.200038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded8cc9-74d4-49d8-a0c8-d341f308ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:53:02.200300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded8cc9-74d4-49d8-a0c8-d341f308ba13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:53:02.201031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded8cc9-74d4-49d8-a0c8-d341f308ba13 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=96d5a3181e4546859e0dd7835abfd4b5, user.id=666cfdf9ef714f5faa84cb34910c2d23) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:53:02.201329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded8cc9-74d4-49d8-a0c8-d341f308ba13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:53:02.203009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded8cc9-74d4-49d8-a0c8-d341f308ba13 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:53:02.208456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 604/1200] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:53:02 2022] GET /identity/v3/role_assignments?user.id=666cfdf9ef714f5faa84cb34910c2d23&scope.project.id=96d5a3181e4546859e0dd7835abfd4b5&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:53:12.057022 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=22 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:53:12.205984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:14.520923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543fba1-9d38-408d-92be-a23ad920ab52 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:14.521519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543fba1-9d38-408d-92be-a23ad920ab52 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:14.521614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543fba1-9d38-408d-92be-a23ad920ab52 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:14.522897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 597/1201] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:14.528438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dc3b5-5af1-46f6-a0e1-8b3c9e452eb9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:14.532406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dc3b5-5af1-46f6-a0e1-8b3c9e452eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:14.532406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dc3b5-5af1-46f6-a0e1-8b3c9e452eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:14.546624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7d1dc3b5-5af1-46f6-a0e1-8b3c9e452eb9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:14.547768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7d1dc3b5-5af1-46f6-a0e1-8b3c9e452eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:14.613336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 605/1202] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:54:14.638716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543fba1-9d38-408d-92be-a23ad920ab52 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:14.741107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543fba1-9d38-408d-92be-a23ad920ab52 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:14.773474 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03b1b4-2db8-4bd1-9760-a7d7073cac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:14.774022 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03b1b4-2db8-4bd1-9760-a7d7073cac3f None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:14.774293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03b1b4-2db8-4bd1-9760-a7d7073cac3f None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:14.774557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03b1b4-2db8-4bd1-9760-a7d7073cac3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:14.784894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03b1b4-2db8-4bd1-9760-a7d7073cac3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:14.786323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03b1b4-2db8-4bd1-9760-a7d7073cac3f None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-5f03b1b4-2db8-4bd1-9760-a7d7073cac3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:14.801698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:14.803356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:14.804337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:14.804337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:14.804337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:14.804337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 598/1203] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:14.812088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dc3b5-5af1-46f6-a0e1-8b3c9e452eb9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:14.908331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dc3b5-5af1-46f6-a0e1-8b3c9e452eb9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:14.937014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:14.942659 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5eed8-70fa-4b7a-bcd1-f5e4d82c7430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:14.943272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5eed8-70fa-4b7a-bcd1-f5e4d82c7430 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:14.943544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5eed8-70fa-4b7a-bcd1-f5e4d82c7430 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:14.943809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5eed8-70fa-4b7a-bcd1-f5e4d82c7430 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:14.944437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5eed8-70fa-4b7a-bcd1-f5e4d82c7430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:14.944765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5eed8-70fa-4b7a-bcd1-f5e4d82c7430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:14.946221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5eed8-70fa-4b7a-bcd1-f5e4d82c7430 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:14.957160 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 606/1204] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:14.963108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03b1b4-2db8-4bd1-9760-a7d7073cac3f None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:15.017335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:15.063468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03b1b4-2db8-4bd1-9760-a7d7073cac3f None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:15.096716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac4912-e87c-4659-a0b1-98ccafca30de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:15.097308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac4912-e87c-4659-a0b1-98ccafca30de None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:15.097586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac4912-e87c-4659-a0b1-98ccafca30de None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:15.097848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac4912-e87c-4659-a0b1-98ccafca30de None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:15.098484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac4912-e87c-4659-a0b1-98ccafca30de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:15.100551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac4912-e87c-4659-a0b1-98ccafca30de None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:15.189659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 599/1205] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 17:54:14 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:54:17.225773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123c70-82d6-4135-a3e8-b6d30252812e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:17.226293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123c70-82d6-4135-a3e8-b6d30252812e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:17.226293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123c70-82d6-4135-a3e8-b6d30252812e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:17.227581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 607/1206] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:17.234091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9a5b-a7a1-435f-84d6-3dd8aaae0f04 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:17.234545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9a5b-a7a1-435f-84d6-3dd8aaae0f04 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:17.234793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9a5b-a7a1-435f-84d6-3dd8aaae0f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:17.253374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c26f9a5b-a7a1-435f-84d6-3dd8aaae0f04 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:17.254543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c26f9a5b-a7a1-435f-84d6-3dd8aaae0f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:17.318615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 600/1207] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:54:17.342692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50123c70-82d6-4135-a3e8-b6d30252812e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:17.445823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50123c70-82d6-4135-a3e8-b6d30252812e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:17.479504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec69708-2bc2-4377-aeb8-525465359931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:17.480055 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec69708-2bc2-4377-aeb8-525465359931 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:17.480331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec69708-2bc2-4377-aeb8-525465359931 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:17.480679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec69708-2bc2-4377-aeb8-525465359931 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:17.490952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec69708-2bc2-4377-aeb8-525465359931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-3ec69708-2bc2-4377-aeb8-525465359931 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:54:17.495636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:17.496778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:17.497666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 608/1208] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:54:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:17.503047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f9a5b-a7a1-435f-84d6-3dd8aaae0f04 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:17.547509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:17.605734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f9a5b-a7a1-435f-84d6-3dd8aaae0f04 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:17.640056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a03cc2-afd3-469c-81ad-9158ed5cf937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:17.640680 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a03cc2-afd3-469c-81ad-9158ed5cf937 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:17.640955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a03cc2-afd3-469c-81ad-9158ed5cf937 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:17.641248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a03cc2-afd3-469c-81ad-9158ed5cf937 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:17.641942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a03cc2-afd3-469c-81ad-9158ed5cf937 None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:17.642196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a03cc2-afd3-469c-81ad-9158ed5cf937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:17.643589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a03cc2-afd3-469c-81ad-9158ed5cf937 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:17.651895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 601/1209] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:54:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:17.657813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec69708-2bc2-4377-aeb8-525465359931 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:17.707726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:17.758437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec69708-2bc2-4377-aeb8-525465359931 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:17.793895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7b3d7-bc62-48a4-974e-f28f287faa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:17.794505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7b3d7-bc62-48a4-974e-f28f287faa4f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:17.794777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7b3d7-bc62-48a4-974e-f28f287faa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:17.795042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7b3d7-bc62-48a4-974e-f28f287faa4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:17.805878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7b3d7-bc62-48a4-974e-f28f287faa4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:17.807321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7b3d7-bc62-48a4-974e-f28f287faa4f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-61c7b3d7-bc62-48a4-974e-f28f287faa4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:17.821698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:17.822837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:17.828870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:17.828870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:17.828870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:17.828870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 609/1210] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:17.829152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a03cc2-afd3-469c-81ad-9158ed5cf937 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:17.924633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:17.927521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a03cc2-afd3-469c-81ad-9158ed5cf937 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:17.962690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0576876-5262-494c-b22f-eead9943e24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:17.963280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0576876-5262-494c-b22f-eead9943e24e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:17.963552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0576876-5262-494c-b22f-eead9943e24e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:17.963817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0576876-5262-494c-b22f-eead9943e24e None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:17.964442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0576876-5262-494c-b22f-eead9943e24e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:17.964763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0576876-5262-494c-b22f-eead9943e24e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:17.966123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0576876-5262-494c-b22f-eead9943e24e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:17.977543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 602/1211] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:17.983203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7b3d7-bc62-48a4-974e-f28f287faa4f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:18.080620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7b3d7-bc62-48a4-974e-f28f287faa4f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:18.082123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:18.112032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1637109-0167-4ef9-ab0b-1f83d62620be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:18.112667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1637109-0167-4ef9-ab0b-1f83d62620be None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:18.112939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1637109-0167-4ef9-ab0b-1f83d62620be None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:18.113201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1637109-0167-4ef9-ab0b-1f83d62620be None admin] PATH_INFO: `/v3/users/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:18.120844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1637109-0167-4ef9-ab0b-1f83d62620be None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:18.122283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1637109-0167-4ef9-ab0b-1f83d62620be None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-b1637109-0167-4ef9-ab0b-1f83d62620be None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:54:18.134688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:18.135852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:18.137068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 610/1212] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:18.141626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0576876-5262-494c-b22f-eead9943e24e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:18.237637 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0576876-5262-494c-b22f-eead9943e24e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:18.271830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd07e28-10cf-40a9-8d3e-b7b04517bcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:18.272420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd07e28-10cf-40a9-8d3e-b7b04517bcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:18.272725 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd07e28-10cf-40a9-8d3e-b7b04517bcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:18.272991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd07e28-10cf-40a9-8d3e-b7b04517bcc5 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:18.273703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd07e28-10cf-40a9-8d3e-b7b04517bcc5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:18.273999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd07e28-10cf-40a9-8d3e-b7b04517bcc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:18.275402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd07e28-10cf-40a9-8d3e-b7b04517bcc5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:18.291818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 603/1213] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 17:54:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:18.297640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1637109-0167-4ef9-ab0b-1f83d62620be None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:18.396839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1637109-0167-4ef9-ab0b-1f83d62620be None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:18.430744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafb992-9ac9-4d1a-8376-0fac0e8cdc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:18.431343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafb992-9ac9-4d1a-8376-0fac0e8cdc3b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:18.431615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafb992-9ac9-4d1a-8376-0fac0e8cdc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:18.431876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafb992-9ac9-4d1a-8376-0fac0e8cdc3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:18.460622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:18.496983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafb992-9ac9-4d1a-8376-0fac0e8cdc3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:18.498464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafb992-9ac9-4d1a-8376-0fac0e8cdc3b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:18.595848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 611/1214] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:54:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:54:18.602271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd07e28-10cf-40a9-8d3e-b7b04517bcc5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:18.621165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:18.696371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd07e28-10cf-40a9-8d3e-b7b04517bcc5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:18.730695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66af46-200f-4058-b385-511718520555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:18.731317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66af46-200f-4058-b385-511718520555 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:18.731586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66af46-200f-4058-b385-511718520555 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:18.731874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66af46-200f-4058-b385-511718520555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:18.742438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66af46-200f-4058-b385-511718520555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:18.743861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66af46-200f-4058-b385-511718520555 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1e66af46-200f-4058-b385-511718520555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:18.757962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:18.759470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:18.760607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:18.760607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:18.760607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:18.760607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 604/1215] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:18.764773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafb992-9ac9-4d1a-8376-0fac0e8cdc3b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:18.824825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:18.862636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafb992-9ac9-4d1a-8376-0fac0e8cdc3b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:18.895003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e37a5-8bcc-4bb7-8946-e52dc58c0b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:18.895564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e37a5-8bcc-4bb7-8946-e52dc58c0b2b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:18.896448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e37a5-8bcc-4bb7-8946-e52dc58c0b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:18.896758 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e37a5-8bcc-4bb7-8946-e52dc58c0b2b None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:18.897472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e37a5-8bcc-4bb7-8946-e52dc58c0b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:18.897763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e37a5-8bcc-4bb7-8946-e52dc58c0b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:18.899124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e37a5-8bcc-4bb7-8946-e52dc58c0b2b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:18.910466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 612/1216] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:18.916824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66af46-200f-4058-b385-511718520555 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:18.984467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:19.012827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66af46-200f-4058-b385-511718520555 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:19.045857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9625e2-902b-4dba-ae87-53d38cd96440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:19.046465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9625e2-902b-4dba-ae87-53d38cd96440 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:19.046737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9625e2-902b-4dba-ae87-53d38cd96440 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:19.047002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9625e2-902b-4dba-ae87-53d38cd96440 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:19.053631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9625e2-902b-4dba-ae87-53d38cd96440 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:19.055350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9625e2-902b-4dba-ae87-53d38cd96440 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-fe9625e2-902b-4dba-ae87-53d38cd96440 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:54:19.066598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:19.067683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:19.073631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:54:19.073631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:54:19.073631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:54:19.073631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:54:19.073631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:54:19.073631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:54:19.073631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:19.073631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:19.073631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 605/1217] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:54:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:19.074272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e37a5-8bcc-4bb7-8946-e52dc58c0b2b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:19.173189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e37a5-8bcc-4bb7-8946-e52dc58c0b2b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:19.194948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:19.206076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cb7d0-3e54-4613-9fce-44fdccba48b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:19.206674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cb7d0-3e54-4613-9fce-44fdccba48b4 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:19.206966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cb7d0-3e54-4613-9fce-44fdccba48b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:19.207216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cb7d0-3e54-4613-9fce-44fdccba48b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:19.207893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9cb7d0-3e54-4613-9fce-44fdccba48b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:19.208194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9cb7d0-3e54-4613-9fce-44fdccba48b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:19.210245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9cb7d0-3e54-4613-9fce-44fdccba48b4 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:19.221793 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 613/1218] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:54:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:19.227917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9625e2-902b-4dba-ae87-53d38cd96440 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:19.326896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9625e2-902b-4dba-ae87-53d38cd96440 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:19.354513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:19.360193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30dac3-35e8-4016-a962-fd390c38da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:19.360800 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30dac3-35e8-4016-a962-fd390c38da19 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:19.361072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30dac3-35e8-4016-a962-fd390c38da19 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:19.361336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30dac3-35e8-4016-a962-fd390c38da19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:19.362041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30dac3-35e8-4016-a962-fd390c38da19 None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=b81bdfeedcba4b0c93f13d14318b2151) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:19.362339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30dac3-35e8-4016-a962-fd390c38da19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:19.363746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30dac3-35e8-4016-a962-fd390c38da19 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:19.368959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 606/1219] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:54:19 2022] GET /identity/v3/role_assignments?user.id=b81bdfeedcba4b0c93f13d14318b2151&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:21.347523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580d628-fcb5-491e-83c2-4344e3a1de4a None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:21.348086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580d628-fcb5-491e-83c2-4344e3a1de4a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:21.348086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580d628-fcb5-491e-83c2-4344e3a1de4a None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:21.349478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 614/1220] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:21.354489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb4826-1ba5-4ab3-b546-8215c182114a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:21.354864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb4826-1ba5-4ab3-b546-8215c182114a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:21.355025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb4826-1ba5-4ab3-b546-8215c182114a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:21.373052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-62bb4826-1ba5-4ab3-b546-8215c182114a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:21.374217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-62bb4826-1ba5-4ab3-b546-8215c182114a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:21.440356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 607/1221] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:54:21.462853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580d628-fcb5-491e-83c2-4344e3a1de4a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:21.561791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580d628-fcb5-491e-83c2-4344e3a1de4a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:21.598482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6bffe-4b73-46d1-b9e6-848ef6a55707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:21.599524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6bffe-4b73-46d1-b9e6-848ef6a55707 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:21.599797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6bffe-4b73-46d1-b9e6-848ef6a55707 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:21.600062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6bffe-4b73-46d1-b9e6-848ef6a55707 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:21.611962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6bffe-4b73-46d1-b9e6-848ef6a55707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-3fb6bffe-4b73-46d1-b9e6-848ef6a55707 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:54:21.617518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:21.618739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:21.620054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 615/1222] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:54:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:21.624673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb4826-1ba5-4ab3-b546-8215c182114a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:21.723464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb4826-1ba5-4ab3-b546-8215c182114a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:21.747160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:21.757493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49426a4-a110-48d4-a0f3-537d16a286df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:21.758089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49426a4-a110-48d4-a0f3-537d16a286df None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:21.758364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49426a4-a110-48d4-a0f3-537d16a286df None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:21.758627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49426a4-a110-48d4-a0f3-537d16a286df None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:21.759274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49426a4-a110-48d4-a0f3-537d16a286df None admin] RBAC: Adding query filter params (name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:21.759569 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49426a4-a110-48d4-a0f3-537d16a286df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:21.760956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49426a4-a110-48d4-a0f3-537d16a286df None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:21.768704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 608/1223] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:54:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:21.775008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6bffe-4b73-46d1-b9e6-848ef6a55707 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:21.821463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:21.875842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6bffe-4b73-46d1-b9e6-848ef6a55707 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:21.910183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67dec-7848-435f-810a-c1ca2e92aa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:21.910806 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67dec-7848-435f-810a-c1ca2e92aa09 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:21.911079 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67dec-7848-435f-810a-c1ca2e92aa09 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:21.911458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f67dec-7848-435f-810a-c1ca2e92aa09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:21.925807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67dec-7848-435f-810a-c1ca2e92aa09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:21.927626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f67dec-7848-435f-810a-c1ca2e92aa09 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c6f67dec-7848-435f-810a-c1ca2e92aa09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:21.942926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:21.944177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:21.945356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:21.945356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:21.945356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:21.945356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 616/1224] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:21.950580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49426a4-a110-48d4-a0f3-537d16a286df None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:22.048725 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49426a4-a110-48d4-a0f3-537d16a286df None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:22.081235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277642f-3fce-4072-8e95-6585d1281058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:22.081830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277642f-3fce-4072-8e95-6585d1281058 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:22.082114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277642f-3fce-4072-8e95-6585d1281058 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:22.082368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277642f-3fce-4072-8e95-6585d1281058 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:22.082998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277642f-3fce-4072-8e95-6585d1281058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:22.084052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277642f-3fce-4072-8e95-6585d1281058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:22.085186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277642f-3fce-4072-8e95-6585d1281058 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:22.096358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 609/1225] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:22.102841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67dec-7848-435f-810a-c1ca2e92aa09 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:22.194334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:22.200740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f67dec-7848-435f-810a-c1ca2e92aa09 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:22.234634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657eb47-19b6-447c-b879-77905ad01419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:22.235252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657eb47-19b6-447c-b879-77905ad01419 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:22.235520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657eb47-19b6-447c-b879-77905ad01419 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:22.235782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657eb47-19b6-447c-b879-77905ad01419 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:22.243428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657eb47-19b6-447c-b879-77905ad01419 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:22.244897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657eb47-19b6-447c-b879-77905ad01419 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2657eb47-19b6-447c-b879-77905ad01419 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:54:22.261233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:22.262384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:22.263613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 617/1226] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:22.268252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277642f-3fce-4072-8e95-6585d1281058 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:22.314377 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:22.368762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277642f-3fce-4072-8e95-6585d1281058 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:22.403244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68227f-08a7-475f-b45b-ca18355ed4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:22.403831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68227f-08a7-475f-b45b-ca18355ed4af None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:22.404102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68227f-08a7-475f-b45b-ca18355ed4af None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:22.404362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68227f-08a7-475f-b45b-ca18355ed4af None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:22.405090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68227f-08a7-475f-b45b-ca18355ed4af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:22.405382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68227f-08a7-475f-b45b-ca18355ed4af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:22.406787 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68227f-08a7-475f-b45b-ca18355ed4af None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:22.423226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 610/1227] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 17:54:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:22.428972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657eb47-19b6-447c-b879-77905ad01419 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:22.528156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657eb47-19b6-447c-b879-77905ad01419 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:22.560517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd30da-b2bb-4d68-8edd-13e46cb5890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:22.561133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd30da-b2bb-4d68-8edd-13e46cb5890c None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:22.561409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd30da-b2bb-4d68-8edd-13e46cb5890c None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:22.561678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd30da-b2bb-4d68-8edd-13e46cb5890c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:22.572382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd30da-b2bb-4d68-8edd-13e46cb5890c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:22.573850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd30da-b2bb-4d68-8edd-13e46cb5890c None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-6fcd30da-b2bb-4d68-8edd-13e46cb5890c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:22.587753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:22.588921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:22.590269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:22.590269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:22.590269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:22.590269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 618/1228] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:22.594646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68227f-08a7-475f-b45b-ca18355ed4af None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:22.692290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68227f-08a7-475f-b45b-ca18355ed4af None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:22.695150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:22.726139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fde764-5947-4bd9-bb05-e12064b0d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:22.726736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fde764-5947-4bd9-bb05-e12064b0d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:22.727009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fde764-5947-4bd9-bb05-e12064b0d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:22.727273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fde764-5947-4bd9-bb05-e12064b0d1d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:22.727901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fde764-5947-4bd9-bb05-e12064b0d1d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:22.728194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fde764-5947-4bd9-bb05-e12064b0d1d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:22.729581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fde764-5947-4bd9-bb05-e12064b0d1d4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:22.740762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 611/1229] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:22.746642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd30da-b2bb-4d68-8edd-13e46cb5890c None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:22.756974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:22.844695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd30da-b2bb-4d68-8edd-13e46cb5890c None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:22.876802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d668da2-c532-4042-a0cb-203341112ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:22.877422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d668da2-c532-4042-a0cb-203341112ada None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:22.877695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d668da2-c532-4042-a0cb-203341112ada None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:22.877957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d668da2-c532-4042-a0cb-203341112ada None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:22.885013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d668da2-c532-4042-a0cb-203341112ada None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:22.886446 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d668da2-c532-4042-a0cb-203341112ada None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-2d668da2-c532-4042-a0cb-203341112ada None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:54:22.897459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:22.899949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:22.901152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:54:22.901152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:54:22.901152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:54:22.901152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:54:22.901152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:54:22.901152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:54:22.901152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:22.901152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:22.901152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 619/1230] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:22.904482 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fde764-5947-4bd9-bb05-e12064b0d1d4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:23.001973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fde764-5947-4bd9-bb05-e12064b0d1d4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:23.034162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175af9ca-9599-4ade-b11b-052aa97c9491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:23.034753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175af9ca-9599-4ade-b11b-052aa97c9491 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:23.035026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175af9ca-9599-4ade-b11b-052aa97c9491 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:23.035293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175af9ca-9599-4ade-b11b-052aa97c9491 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:23.035972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175af9ca-9599-4ade-b11b-052aa97c9491 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:23.036816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175af9ca-9599-4ade-b11b-052aa97c9491 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:23.038236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175af9ca-9599-4ade-b11b-052aa97c9491 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:23.049711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 612/1231] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:23.056374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d668da2-c532-4042-a0cb-203341112ada None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:23.121453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:23.153449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d668da2-c532-4042-a0cb-203341112ada None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:23.185347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b248c80-5a74-4234-81e5-97dc55aef8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:23.185972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b248c80-5a74-4234-81e5-97dc55aef8a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:23.186242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b248c80-5a74-4234-81e5-97dc55aef8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:23.186529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b248c80-5a74-4234-81e5-97dc55aef8a1 None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/b81bdfeedcba4b0c93f13d14318b2151/roles/2978a8a56cd84b4b852dc6738c5c54c3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:23.200062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b248c80-5a74-4234-81e5-97dc55aef8a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=b81bdfeedcba4b0c93f13d14318b2151, role_id=2978a8a56cd84b4b852dc6738c5c54c3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:23.201787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b248c80-5a74-4234-81e5-97dc55aef8a1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:23.278610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 620/1232] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:54:23 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/b81bdfeedcba4b0c93f13d14318b2151/roles/2978a8a56cd84b4b852dc6738c5c54c3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:54:25.248786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fd087-4557-4815-9a1d-cb45686b0c94 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:25.249284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fd087-4557-4815-9a1d-cb45686b0c94 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:25.249284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fd087-4557-4815-9a1d-cb45686b0c94 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:25.250561 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 613/1233] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:25.256361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f317-6ee9-4994-bb2a-2e9f76531232 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:25.256810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f317-6ee9-4994-bb2a-2e9f76531232 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:25.256965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f317-6ee9-4994-bb2a-2e9f76531232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:25.275102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-20e0f317-6ee9-4994-bb2a-2e9f76531232 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:25.276352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-20e0f317-6ee9-4994-bb2a-2e9f76531232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:25.354705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 621/1234] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:54:25.377450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fd087-4557-4815-9a1d-cb45686b0c94 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:25.403950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:25.480011 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fd087-4557-4815-9a1d-cb45686b0c94 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:25.513405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e926e-73de-47c2-862b-4b6158f57030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:25.514052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e926e-73de-47c2-862b-4b6158f57030 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:25.514322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e926e-73de-47c2-862b-4b6158f57030 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:25.514584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e926e-73de-47c2-862b-4b6158f57030 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:25.525240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e926e-73de-47c2-862b-4b6158f57030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-241e926e-73de-47c2-862b-4b6158f57030 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:54:25.531238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:25.532350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:25.533426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 614/1235] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 17:54:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:25.537246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0f317-6ee9-4994-bb2a-2e9f76531232 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:25.583570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:25.637860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0f317-6ee9-4994-bb2a-2e9f76531232 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:25.670373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279e180-4b1e-4ac5-816d-c83b00873965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:25.670950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279e180-4b1e-4ac5-816d-c83b00873965 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:25.671222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279e180-4b1e-4ac5-816d-c83b00873965 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:25.671487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279e180-4b1e-4ac5-816d-c83b00873965 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:25.672146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279e180-4b1e-4ac5-816d-c83b00873965 None admin] RBAC: Adding query filter params (name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:25.672440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279e180-4b1e-4ac5-816d-c83b00873965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:25.673832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279e180-4b1e-4ac5-816d-c83b00873965 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:25.681890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 622/1236] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 17:54:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:25.687856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e926e-73de-47c2-862b-4b6158f57030 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:25.772088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:25.787233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e926e-73de-47c2-862b-4b6158f57030 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:25.821219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a520f-5d20-4760-abf5-d456d3bace56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:25.821835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a520f-5d20-4760-abf5-d456d3bace56 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:25.822109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a520f-5d20-4760-abf5-d456d3bace56 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:25.822373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a520f-5d20-4760-abf5-d456d3bace56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:25.832181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a520f-5d20-4760-abf5-d456d3bace56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:25.833683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a520f-5d20-4760-abf5-d456d3bace56 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-b02a520f-5d20-4760-abf5-d456d3bace56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:25.847691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:25.848852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:25.849922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:25.849922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:25.849922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:25.849922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 615/1237] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:25.854932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279e180-4b1e-4ac5-816d-c83b00873965 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:25.952949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279e180-4b1e-4ac5-816d-c83b00873965 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:25.962590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:25.986675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22ed06-c613-4aea-8b07-59ac1fdaae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:25.987271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22ed06-c613-4aea-8b07-59ac1fdaae47 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:25.987545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22ed06-c613-4aea-8b07-59ac1fdaae47 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:25.987810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22ed06-c613-4aea-8b07-59ac1fdaae47 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:25.988437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22ed06-c613-4aea-8b07-59ac1fdaae47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:25.988758 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22ed06-c613-4aea-8b07-59ac1fdaae47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:25.990685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22ed06-c613-4aea-8b07-59ac1fdaae47 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:26.002028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 623/1238] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:26.007755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a520f-5d20-4760-abf5-d456d3bace56 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:26.107856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a520f-5d20-4760-abf5-d456d3bace56 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:26.141958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce660c9-97c5-4bbd-b0cb-89616fd9cc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:26.142577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce660c9-97c5-4bbd-b0cb-89616fd9cc97 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:26.142851 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce660c9-97c5-4bbd-b0cb-89616fd9cc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:26.143121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce660c9-97c5-4bbd-b0cb-89616fd9cc97 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:26.145734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:26.151380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce660c9-97c5-4bbd-b0cb-89616fd9cc97 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:26.152834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce660c9-97c5-4bbd-b0cb-89616fd9cc97 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-3ce660c9-97c5-4bbd-b0cb-89616fd9cc97 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:54:26.164202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:26.165362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:26.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 616/1239] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:26.171077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22ed06-c613-4aea-8b07-59ac1fdaae47 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:26.268792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22ed06-c613-4aea-8b07-59ac1fdaae47 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:26.302788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dddb5-71e4-4547-a744-2fe068155f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:26.303384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dddb5-71e4-4547-a744-2fe068155f99 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:26.303658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dddb5-71e4-4547-a744-2fe068155f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:26.303926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dddb5-71e4-4547-a744-2fe068155f99 None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:26.304669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dddb5-71e4-4547-a744-2fe068155f99 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:26.304969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dddb5-71e4-4547-a744-2fe068155f99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:26.306395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dddb5-71e4-4547-a744-2fe068155f99 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:26.323487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 624/1240] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 17:54:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:26.329556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce660c9-97c5-4bbd-b0cb-89616fd9cc97 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:26.428312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce660c9-97c5-4bbd-b0cb-89616fd9cc97 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:26.461245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20d49f-b07e-4b49-972b-a96bd91bbda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:26.461856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20d49f-b07e-4b49-972b-a96bd91bbda5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:26.462108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20d49f-b07e-4b49-972b-a96bd91bbda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:26.462371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20d49f-b07e-4b49-972b-a96bd91bbda5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:26.521065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:26.527161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20d49f-b07e-4b49-972b-a96bd91bbda5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:26.528629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20d49f-b07e-4b49-972b-a96bd91bbda5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:26.627727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 617/1241] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:54:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:54:26.633638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dddb5-71e4-4547-a744-2fe068155f99 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:26.660989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:26.732106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dddb5-71e4-4547-a744-2fe068155f99 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:26.764751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4b059-fc04-4627-bd73-7bfc54c820f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:26.765356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4b059-fc04-4627-bd73-7bfc54c820f7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:26.765626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4b059-fc04-4627-bd73-7bfc54c820f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:26.765888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4b059-fc04-4627-bd73-7bfc54c820f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:26.776136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4b059-fc04-4627-bd73-7bfc54c820f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:26.777589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4b059-fc04-4627-bd73-7bfc54c820f7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-dfa4b059-fc04-4627-bd73-7bfc54c820f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:26.791732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:26.792896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:26.794002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:26.794002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:26.794002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:26.794002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 625/1242] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:26.798560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20d49f-b07e-4b49-972b-a96bd91bbda5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:26.891926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:26.898570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20d49f-b07e-4b49-972b-a96bd91bbda5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:26.931510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6950809-1fb0-480d-a7b4-2956dbf6705a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:26.932099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6950809-1fb0-480d-a7b4-2956dbf6705a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:26.932374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6950809-1fb0-480d-a7b4-2956dbf6705a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:26.932665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6950809-1fb0-480d-a7b4-2956dbf6705a None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:26.933296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6950809-1fb0-480d-a7b4-2956dbf6705a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:26.933698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6950809-1fb0-480d-a7b4-2956dbf6705a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:26.935047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6950809-1fb0-480d-a7b4-2956dbf6705a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:26.946262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 618/1243] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:26.952284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4b059-fc04-4627-bd73-7bfc54c820f7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:27.030735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:27.051376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4b059-fc04-4627-bd73-7bfc54c820f7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:27.084614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43748a-a19b-4846-b1bc-e4d4ac372f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:27.085231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43748a-a19b-4846-b1bc-e4d4ac372f42 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:27.085502 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43748a-a19b-4846-b1bc-e4d4ac372f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:27.085768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43748a-a19b-4846-b1bc-e4d4ac372f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:27.092070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43748a-a19b-4846-b1bc-e4d4ac372f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:27.093571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43748a-a19b-4846-b1bc-e4d4ac372f42 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-5b43748a-a19b-4846-b1bc-e4d4ac372f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:54:27.105135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:27.110881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:27.112027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:54:27.112027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:54:27.112027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:54:27.112027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:54:27.112027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:54:27.112027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:54:27.112027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:27.112027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:27.112027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 626/1244] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:54:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:54:27.112649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6950809-1fb0-480d-a7b4-2956dbf6705a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:27.213244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6950809-1fb0-480d-a7b4-2956dbf6705a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:27.245967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca961214-ae79-4529-b28a-0b1bc7b53a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:27.247628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca961214-ae79-4529-b28a-0b1bc7b53a3d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:27.247628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca961214-ae79-4529-b28a-0b1bc7b53a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:27.247628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca961214-ae79-4529-b28a-0b1bc7b53a3d None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:27.247932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca961214-ae79-4529-b28a-0b1bc7b53a3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:27.248056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca961214-ae79-4529-b28a-0b1bc7b53a3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:27.249509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca961214-ae79-4529-b28a-0b1bc7b53a3d None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:27.262758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 619/1245] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:27.268995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43748a-a19b-4846-b1bc-e4d4ac372f42 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:27.367277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43748a-a19b-4846-b1bc-e4d4ac372f42 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:27.400159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148fd43-62ed-4836-bee3-69bfc7a35f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:27.400784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148fd43-62ed-4836-bee3-69bfc7a35f2d None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:27.401266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148fd43-62ed-4836-bee3-69bfc7a35f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:27.401538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148fd43-62ed-4836-bee3-69bfc7a35f2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:27.402336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148fd43-62ed-4836-bee3-69bfc7a35f2d None admin] RBAC: Adding query filter params (role.id=2978a8a56cd84b4b852dc6738c5c54c3, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=b81bdfeedcba4b0c93f13d14318b2151) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:27.402640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148fd43-62ed-4836-bee3-69bfc7a35f2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:27.404078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148fd43-62ed-4836-bee3-69bfc7a35f2d None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:27.409985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 627/1246] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:54:27 2022] GET /identity/v3/role_assignments?user.id=b81bdfeedcba4b0c93f13d14318b2151&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=2978a8a56cd84b4b852dc6738c5c54c3 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:29.459941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b876ccf-6bf3-4650-b835-09a393eccb33 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:29.460494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b876ccf-6bf3-4650-b835-09a393eccb33 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:29.460494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b876ccf-6bf3-4650-b835-09a393eccb33 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:29.461823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 620/1247] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:54:29.467252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63e891-3b66-4c21-914c-c6b6fce10342 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:29.467697 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63e891-3b66-4c21-914c-c6b6fce10342 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:29.467805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63e891-3b66-4c21-914c-c6b6fce10342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:29.474878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:29.487115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ed63e891-3b66-4c21-914c-c6b6fce10342 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:29.488306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ed63e891-3b66-4c21-914c-c6b6fce10342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:29.552416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 628/1248] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:54:29.575620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b876ccf-6bf3-4650-b835-09a393eccb33 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:29.594102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:29.674528 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b876ccf-6bf3-4650-b835-09a393eccb33 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:29.710091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ce199-6445-45d6-bcd9-ac61e135a87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:29.710641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ce199-6445-45d6-bcd9-ac61e135a87d None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:29.710905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ce199-6445-45d6-bcd9-ac61e135a87d None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:29.711171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ce199-6445-45d6-bcd9-ac61e135a87d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:29.720961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ce199-6445-45d6-bcd9-ac61e135a87d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-564ce199-6445-45d6-bcd9-ac61e135a87d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:54:29.725616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:29.731588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:29.732683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 621/1249] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:54:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:29.734199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63e891-3b66-4c21-914c-c6b6fce10342 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:29.831126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63e891-3b66-4c21-914c-c6b6fce10342 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:29.859745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:29.864523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92644855-ffad-462b-8b1c-d40d9e956dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:29.865115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92644855-ffad-462b-8b1c-d40d9e956dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:29.865388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92644855-ffad-462b-8b1c-d40d9e956dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:29.865655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92644855-ffad-462b-8b1c-d40d9e956dd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:29.866301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92644855-ffad-462b-8b1c-d40d9e956dd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:29.866595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92644855-ffad-462b-8b1c-d40d9e956dd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:29.867947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92644855-ffad-462b-8b1c-d40d9e956dd1 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:29.875675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 629/1250] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:54:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:54:29.882087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ce199-6445-45d6-bcd9-ac61e135a87d None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:29.958841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:29.980590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ce199-6445-45d6-bcd9-ac61e135a87d None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:30.012719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af56c5-29e4-41af-90a6-266b01e44210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:30.013330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af56c5-29e4-41af-90a6-266b01e44210 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:30.013604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af56c5-29e4-41af-90a6-266b01e44210 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:30.013867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af56c5-29e4-41af-90a6-266b01e44210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:30.024737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af56c5-29e4-41af-90a6-266b01e44210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:30.026177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af56c5-29e4-41af-90a6-266b01e44210 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-96af56c5-29e4-41af-90a6-266b01e44210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:30.039873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:30.042826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:30.043982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:30.043982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:30.043982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:30.043982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 622/1251] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:30.046909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92644855-ffad-462b-8b1c-d40d9e956dd1 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:30.146095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92644855-ffad-462b-8b1c-d40d9e956dd1 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:30.178328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b636286-d5bf-469e-afbd-372354515f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:30.178938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b636286-d5bf-469e-afbd-372354515f84 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:30.179215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b636286-d5bf-469e-afbd-372354515f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:30.179484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b636286-d5bf-469e-afbd-372354515f84 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:30.180112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b636286-d5bf-469e-afbd-372354515f84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:30.180410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b636286-d5bf-469e-afbd-372354515f84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:30.181809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b636286-d5bf-469e-afbd-372354515f84 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:30.193609 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 630/1252] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:30.199455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af56c5-29e4-41af-90a6-266b01e44210 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:30.296904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af56c5-29e4-41af-90a6-266b01e44210 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:30.323406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:30.331291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab032403-8217-4a2b-9235-38e3f6451a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:30.331906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab032403-8217-4a2b-9235-38e3f6451a94 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:30.332180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab032403-8217-4a2b-9235-38e3f6451a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:30.332444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab032403-8217-4a2b-9235-38e3f6451a94 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:30.339610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab032403-8217-4a2b-9235-38e3f6451a94 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:30.341133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab032403-8217-4a2b-9235-38e3f6451a94 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-ab032403-8217-4a2b-9235-38e3f6451a94 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:54:30.351877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:30.353055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:30.354251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 623/1253] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:30.358892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b636286-d5bf-469e-afbd-372354515f84 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:30.413766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:30.456461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b636286-d5bf-469e-afbd-372354515f84 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:30.490365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fa3ce-c61d-4bd3-ac8d-eaf21e07804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:30.490945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fa3ce-c61d-4bd3-ac8d-eaf21e07804f None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:30.491205 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fa3ce-c61d-4bd3-ac8d-eaf21e07804f None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:30.491468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fa3ce-c61d-4bd3-ac8d-eaf21e07804f None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:30.492164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fa3ce-c61d-4bd3-ac8d-eaf21e07804f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:30.492455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fa3ce-c61d-4bd3-ac8d-eaf21e07804f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:30.493927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fa3ce-c61d-4bd3-ac8d-eaf21e07804f None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:30.511848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 631/1254] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 17:54:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:30.517781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab032403-8217-4a2b-9235-38e3f6451a94 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:30.615366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab032403-8217-4a2b-9235-38e3f6451a94 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:30.647959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f66f3-2a52-408b-a804-e5b5e0e9bfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:30.648563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f66f3-2a52-408b-a804-e5b5e0e9bfde None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:30.648837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f66f3-2a52-408b-a804-e5b5e0e9bfde None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:30.649101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f66f3-2a52-408b-a804-e5b5e0e9bfde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:30.703066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:30.714893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f66f3-2a52-408b-a804-e5b5e0e9bfde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:30.716342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f66f3-2a52-408b-a804-e5b5e0e9bfde None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:30.810462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 624/1255] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:54:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:54:30.816288 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fa3ce-c61d-4bd3-ac8d-eaf21e07804f None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:30.913091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fa3ce-c61d-4bd3-ac8d-eaf21e07804f None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:30.926040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:30.946220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cf6dc-6b36-4142-bee2-d46de56be77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:30.946823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cf6dc-6b36-4142-bee2-d46de56be77b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:30.947090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cf6dc-6b36-4142-bee2-d46de56be77b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:30.947348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cf6dc-6b36-4142-bee2-d46de56be77b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:30.957083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cf6dc-6b36-4142-bee2-d46de56be77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:30.959020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cf6dc-6b36-4142-bee2-d46de56be77b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-c82cf6dc-6b36-4142-bee2-d46de56be77b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:30.972741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:30.974467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:30.975766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:30.975766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:30.975766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:30.975766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 632/1256] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:30.980206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f66f3-2a52-408b-a804-e5b5e0e9bfde None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:31.067647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:31.079355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f66f3-2a52-408b-a804-e5b5e0e9bfde None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:31.112254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577cef1c-fcd6-43bc-b68c-1a344a357013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:31.112866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cef1c-fcd6-43bc-b68c-1a344a357013 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:31.113135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cef1c-fcd6-43bc-b68c-1a344a357013 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:31.113395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cef1c-fcd6-43bc-b68c-1a344a357013 None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:31.114039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577cef1c-fcd6-43bc-b68c-1a344a357013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:31.114329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577cef1c-fcd6-43bc-b68c-1a344a357013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:31.115664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577cef1c-fcd6-43bc-b68c-1a344a357013 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:31.126226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 625/1257] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:31.131926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cf6dc-6b36-4142-bee2-d46de56be77b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:31.239099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cf6dc-6b36-4142-bee2-d46de56be77b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:31.273332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565165d-b259-4151-bc63-38ca3c64c643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:31.273948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565165d-b259-4151-bc63-38ca3c64c643 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:31.274233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565165d-b259-4151-bc63-38ca3c64c643 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:31.274492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565165d-b259-4151-bc63-38ca3c64c643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:31.281045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565165d-b259-4151-bc63-38ca3c64c643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:31.283237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565165d-b259-4151-bc63-38ca3c64c643 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-0565165d-b259-4151-bc63-38ca3c64c643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:54:31.293373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:31.294441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:31.295552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:54:31.295552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:54:31.295552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:54:31.295552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:54:31.295552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:54:31.295552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:54:31.295552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:31.295552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:31.295552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 633/1258] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:31.300443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577cef1c-fcd6-43bc-b68c-1a344a357013 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:31.398501 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577cef1c-fcd6-43bc-b68c-1a344a357013 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:31.430793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f23a7-1323-4cfe-b3f7-97998c3fab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:31.431409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f23a7-1323-4cfe-b3f7-97998c3fab72 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:31.431685 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f23a7-1323-4cfe-b3f7-97998c3fab72 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:31.431952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f23a7-1323-4cfe-b3f7-97998c3fab72 None admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:31.432656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f23a7-1323-4cfe-b3f7-97998c3fab72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:31.432954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f23a7-1323-4cfe-b3f7-97998c3fab72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:31.434390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f23a7-1323-4cfe-b3f7-97998c3fab72 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:31.451490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 626/1259] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:54:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:31.457773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565165d-b259-4151-bc63-38ca3c64c643 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:31.483465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:31.556490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565165d-b259-4151-bc63-38ca3c64c643 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:31.589605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccabb474-b055-49ed-bd3e-9e2c067c8f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:31.590212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabb474-b055-49ed-bd3e-9e2c067c8f31 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:31.590483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabb474-b055-49ed-bd3e-9e2c067c8f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:31.590746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabb474-b055-49ed-bd3e-9e2c067c8f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:31.591473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccabb474-b055-49ed-bd3e-9e2c067c8f31 None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=b81bdfeedcba4b0c93f13d14318b2151) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:31.591770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccabb474-b055-49ed-bd3e-9e2c067c8f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:31.593212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccabb474-b055-49ed-bd3e-9e2c067c8f31 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:31.598437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 634/1260] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:54:31 2022] GET /identity/v3/role_assignments?user.id=b81bdfeedcba4b0c93f13d14318b2151&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:33.589539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48519152-7546-4ecf-8bde-9861dd7cfb61 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:33.590138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48519152-7546-4ecf-8bde-9861dd7cfb61 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:33.590138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48519152-7546-4ecf-8bde-9861dd7cfb61 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:33.591387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 627/1261] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:33.596610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88dfd5-c381-4909-a854-0dde9a0a798f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:33.597047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88dfd5-c381-4909-a854-0dde9a0a798f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:33.600252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88dfd5-c381-4909-a854-0dde9a0a798f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:33.616152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0a88dfd5-c381-4909-a854-0dde9a0a798f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:33.617359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0a88dfd5-c381-4909-a854-0dde9a0a798f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:33.682090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 635/1262] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:54:33.704305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48519152-7546-4ecf-8bde-9861dd7cfb61 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:33.715138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:33.805497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48519152-7546-4ecf-8bde-9861dd7cfb61 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:33.838827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeacd5-2270-466c-93a8-88b365ab78c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:33.839472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeacd5-2270-466c-93a8-88b365ab78c7 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:33.839791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeacd5-2270-466c-93a8-88b365ab78c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:33.840009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeacd5-2270-466c-93a8-88b365ab78c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:33.850309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdeacd5-2270-466c-93a8-88b365ab78c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-5bdeacd5-2270-466c-93a8-88b365ab78c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:54:33.854975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:33.856077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:33.856988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 628/1263] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:54:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:33.862243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88dfd5-c381-4909-a854-0dde9a0a798f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:33.879960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:33.960272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88dfd5-c381-4909-a854-0dde9a0a798f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:33.994481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1e232-8972-4cc2-8672-2f2aeddf4bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:33.995073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1e232-8972-4cc2-8672-2f2aeddf4bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:33.995347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1e232-8972-4cc2-8672-2f2aeddf4bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:33.995612 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1e232-8972-4cc2-8672-2f2aeddf4bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:33.996264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1e232-8972-4cc2-8672-2f2aeddf4bb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:33.996587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1e232-8972-4cc2-8672-2f2aeddf4bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:33.997967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1e232-8972-4cc2-8672-2f2aeddf4bb8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:34.005850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 636/1264] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:54:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:34.011910 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeacd5-2270-466c-93a8-88b365ab78c7 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:34.078164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:34.111636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeacd5-2270-466c-93a8-88b365ab78c7 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:34.145622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b017c6e-ec0f-42b0-b186-861d374450c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:34.146226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b017c6e-ec0f-42b0-b186-861d374450c5 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:34.146495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b017c6e-ec0f-42b0-b186-861d374450c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:34.146756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b017c6e-ec0f-42b0-b186-861d374450c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:34.156644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b017c6e-ec0f-42b0-b186-861d374450c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:34.158074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b017c6e-ec0f-42b0-b186-861d374450c5 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-6b017c6e-ec0f-42b0-b186-861d374450c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:34.172733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:34.174137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:34.175460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:34.175460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:34.175460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:34.175460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 629/1265] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:34.179645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1e232-8972-4cc2-8672-2f2aeddf4bb8 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:34.258744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:34.277408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1e232-8972-4cc2-8672-2f2aeddf4bb8 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:34.308496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6358f6c-7b8f-4da6-9128-247c5954f4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:34.309099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6358f6c-7b8f-4da6-9128-247c5954f4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:34.309372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6358f6c-7b8f-4da6-9128-247c5954f4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:34.309639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6358f6c-7b8f-4da6-9128-247c5954f4b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:34.310262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6358f6c-7b8f-4da6-9128-247c5954f4b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:34.310552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6358f6c-7b8f-4da6-9128-247c5954f4b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:34.311895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6358f6c-7b8f-4da6-9128-247c5954f4b0 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:34.324117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 637/1266] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:34.330304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b017c6e-ec0f-42b0-b186-861d374450c5 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:34.430021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b017c6e-ec0f-42b0-b186-861d374450c5 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:34.449325 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:34.463490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56cab0-9fa6-4699-9e7e-1f2c83036c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:34.464097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56cab0-9fa6-4699-9e7e-1f2c83036c89 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:34.464367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56cab0-9fa6-4699-9e7e-1f2c83036c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:34.464709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56cab0-9fa6-4699-9e7e-1f2c83036c89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:34.471723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56cab0-9fa6-4699-9e7e-1f2c83036c89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:34.473232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56cab0-9fa6-4699-9e7e-1f2c83036c89 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-1a56cab0-9fa6-4699-9e7e-1f2c83036c89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:54:34.483988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:34.485192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:34.486333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 630/1267] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:34.490925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6358f6c-7b8f-4da6-9128-247c5954f4b0 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:34.592090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6358f6c-7b8f-4da6-9128-247c5954f4b0 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:34.624699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698a2bc-2593-4a00-a4a8-236376acbabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:34.625289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698a2bc-2593-4a00-a4a8-236376acbabf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:34.625560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698a2bc-2593-4a00-a4a8-236376acbabf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:34.625824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698a2bc-2593-4a00-a4a8-236376acbabf None admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:34.626530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2698a2bc-2593-4a00-a4a8-236376acbabf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:34.626825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2698a2bc-2593-4a00-a4a8-236376acbabf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:34.628283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2698a2bc-2593-4a00-a4a8-236376acbabf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:34.637344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:34.645822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 638/1268] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 17:54:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:34.651607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56cab0-9fa6-4699-9e7e-1f2c83036c89 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:34.753545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56cab0-9fa6-4699-9e7e-1f2c83036c89 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:34.785773 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de758ffc-4490-4cef-bce0-6c3a7cce3d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:34.786384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de758ffc-4490-4cef-bce0-6c3a7cce3d7e None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:34.786653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de758ffc-4490-4cef-bce0-6c3a7cce3d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:34.786915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de758ffc-4490-4cef-bce0-6c3a7cce3d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:34.797076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de758ffc-4490-4cef-bce0-6c3a7cce3d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:34.798510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de758ffc-4490-4cef-bce0-6c3a7cce3d7e None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-de758ffc-4490-4cef-bce0-6c3a7cce3d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:34.812864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:34.813998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:34.815303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:34.815303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:34.815303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:34.815303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 631/1269] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:34.820309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698a2bc-2593-4a00-a4a8-236376acbabf None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:34.919150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698a2bc-2593-4a00-a4a8-236376acbabf None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:34.952718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dd10d-938b-4816-ac61-911d784b8fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:34.953305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dd10d-938b-4816-ac61-911d784b8fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:34.953577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dd10d-938b-4816-ac61-911d784b8fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:34.953840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dd10d-938b-4816-ac61-911d784b8fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:34.954465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dd10d-938b-4816-ac61-911d784b8fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:34.954755 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dd10d-938b-4816-ac61-911d784b8fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:34.956136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dd10d-938b-4816-ac61-911d784b8fb7 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:34.966895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 639/1270] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:34.972592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de758ffc-4490-4cef-bce0-6c3a7cce3d7e None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:35.004119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:35.074630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de758ffc-4490-4cef-bce0-6c3a7cce3d7e None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:35.108010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb5a1b-98ad-4cc8-94ab-34bbe0e24d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:35.108650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb5a1b-98ad-4cc8-94ab-34bbe0e24d34 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:35.108924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb5a1b-98ad-4cc8-94ab-34bbe0e24d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:35.109193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb5a1b-98ad-4cc8-94ab-34bbe0e24d34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:35.116151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb5a1b-98ad-4cc8-94ab-34bbe0e24d34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:35.117635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb5a1b-98ad-4cc8-94ab-34bbe0e24d34 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-dafb5a1b-98ad-4cc8-94ab-34bbe0e24d34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:54:35.128626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:35.129661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:35.130960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:54:35.130960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:54:35.130960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:54:35.130960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:54:35.130960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:54:35.130960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:54:35.130960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:35.130960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:35.130960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 632/1271] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:35.135591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dd10d-938b-4816-ac61-911d784b8fb7 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:35.193334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:35.234782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dd10d-938b-4816-ac61-911d784b8fb7 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:35.268356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236b6b1-22f6-4ee3-af2b-ff1595074706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:35.268926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236b6b1-22f6-4ee3-af2b-ff1595074706 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:35.269198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236b6b1-22f6-4ee3-af2b-ff1595074706 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:35.269477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236b6b1-22f6-4ee3-af2b-ff1595074706 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:35.270638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236b6b1-22f6-4ee3-af2b-ff1595074706 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:35.270936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236b6b1-22f6-4ee3-af2b-ff1595074706 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:35.272364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236b6b1-22f6-4ee3-af2b-ff1595074706 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:35.284624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 640/1272] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:54:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:35.291393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb5a1b-98ad-4cc8-94ab-34bbe0e24d34 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:35.381979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:35.390412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb5a1b-98ad-4cc8-94ab-34bbe0e24d34 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:35.423510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8527129-420f-4d55-ab57-22616a0fd4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:35.424151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8527129-420f-4d55-ab57-22616a0fd4df None admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:35.424427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8527129-420f-4d55-ab57-22616a0fd4df None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:35.424724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8527129-420f-4d55-ab57-22616a0fd4df None admin] PATH_INFO: `/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/b81bdfeedcba4b0c93f13d14318b2151/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:35.426604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8527129-420f-4d55-ab57-22616a0fd4df None admin] RBAC: Authorizing `identity:create_grant(project_id=35b6705a8d9647d3a9ea2b66510f8add, user_id=b81bdfeedcba4b0c93f13d14318b2151, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:35.428157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8527129-420f-4d55-ab57-22616a0fd4df None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:35.514570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 633/1273] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 17:54:35 2022] PUT /identity/v3/projects/35b6705a8d9647d3a9ea2b66510f8add/users/b81bdfeedcba4b0c93f13d14318b2151/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:54:37.516013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b23772-0ff0-4588-8917-a5fb2e73e185 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:37.516699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b23772-0ff0-4588-8917-a5fb2e73e185 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:37.516699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b23772-0ff0-4588-8917-a5fb2e73e185 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:37.517915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 641/1274] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:37.523666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98b3b1-a732-4d2d-97e2-28790dea6750 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:37.524093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98b3b1-a732-4d2d-97e2-28790dea6750 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:37.524189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98b3b1-a732-4d2d-97e2-28790dea6750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:37.541990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4f98b3b1-a732-4d2d-97e2-28790dea6750 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:37.543178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4f98b3b1-a732-4d2d-97e2-28790dea6750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:37.609790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 634/1275] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:54:37.634101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b23772-0ff0-4588-8917-a5fb2e73e185 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:37.732194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b23772-0ff0-4588-8917-a5fb2e73e185 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:37.749381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:37.766355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ec325-39ec-4ad5-a1be-aad465ccb95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:37.766904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ec325-39ec-4ad5-a1be-aad465ccb95e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:37.767178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ec325-39ec-4ad5-a1be-aad465ccb95e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:37.767445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ec325-39ec-4ad5-a1be-aad465ccb95e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:37.777261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ec325-39ec-4ad5-a1be-aad465ccb95e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f74ec325-39ec-4ad5-a1be-aad465ccb95e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 17:54:37.781864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:37.782983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:37.784006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 642/1276] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 17:54:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:37.789442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98b3b1-a732-4d2d-97e2-28790dea6750 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:37.873168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:37.890907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98b3b1-a732-4d2d-97e2-28790dea6750 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:37.923212 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e74c0-cce7-453b-b0e7-34d524969cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:37.923797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e74c0-cce7-453b-b0e7-34d524969cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:37.924068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e74c0-cce7-453b-b0e7-34d524969cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:37.924333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e74c0-cce7-453b-b0e7-34d524969cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:37.925013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e74c0-cce7-453b-b0e7-34d524969cd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:37.925309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e74c0-cce7-453b-b0e7-34d524969cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:37.927443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e74c0-cce7-453b-b0e7-34d524969cd4 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:37.935088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 635/1277] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 17:54:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:37.940847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ec325-39ec-4ad5-a1be-aad465ccb95e None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:38.040738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ec325-39ec-4ad5-a1be-aad465ccb95e None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:38.073666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ef33-06dc-470c-a4da-b24791141c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:38.074273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ef33-06dc-470c-a4da-b24791141c03 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:38.074543 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ef33-06dc-470c-a4da-b24791141c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:38.074804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ef33-06dc-470c-a4da-b24791141c03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:38.085143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324ef33-06dc-470c-a4da-b24791141c03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:38.091483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324ef33-06dc-470c-a4da-b24791141c03 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-f324ef33-06dc-470c-a4da-b24791141c03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:38.100544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:38.101653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:38.102814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:38.102814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:38.102814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:38.102814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 643/1278] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:38.107535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e74c0-cce7-453b-b0e7-34d524969cd4 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:38.206427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e74c0-cce7-453b-b0e7-34d524969cd4 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:38.240209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5959b4-9166-4953-9cbc-56154fecdbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:38.240898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5959b4-9166-4953-9cbc-56154fecdbfb None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:38.241168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5959b4-9166-4953-9cbc-56154fecdbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:38.241513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5959b4-9166-4953-9cbc-56154fecdbfb None admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:38.242141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5959b4-9166-4953-9cbc-56154fecdbfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:38.242433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5959b4-9166-4953-9cbc-56154fecdbfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:38.243782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5959b4-9166-4953-9cbc-56154fecdbfb None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:38.255110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 636/1279] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:38.261150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ef33-06dc-470c-a4da-b24791141c03 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:38.293412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:38.360450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ef33-06dc-470c-a4da-b24791141c03 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:38.393506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7edde-d60f-49ed-ada2-eb141c6080bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:38.394122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7edde-d60f-49ed-ada2-eb141c6080bf None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:38.394523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7edde-d60f-49ed-ada2-eb141c6080bf None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:38.394795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7edde-d60f-49ed-ada2-eb141c6080bf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:38.402716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7edde-d60f-49ed-ada2-eb141c6080bf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:38.404145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7edde-d60f-49ed-ada2-eb141c6080bf None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-d6a7edde-d60f-49ed-ada2-eb141c6080bf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 17:54:38.415657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:38.417969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:38.418995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 644/1280] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 17:54:38.422701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5959b4-9166-4953-9cbc-56154fecdbfb None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:38.426499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:38.519728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5959b4-9166-4953-9cbc-56154fecdbfb None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:38.554259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6f12c-092c-4c84-bb86-30a4b6804808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:38.554899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6f12c-092c-4c84-bb86-30a4b6804808 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:38.555105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6f12c-092c-4c84-bb86-30a4b6804808 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:38.555366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6f12c-092c-4c84-bb86-30a4b6804808 None admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:38.556070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6f12c-092c-4c84-bb86-30a4b6804808 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:38.556362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6f12c-092c-4c84-bb86-30a4b6804808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:38.557788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6f12c-092c-4c84-bb86-30a4b6804808 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:38.575263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 637/1281] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 17:54:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:38.581332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7edde-d60f-49ed-ada2-eb141c6080bf None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:38.674076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:38.684141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7edde-d60f-49ed-ada2-eb141c6080bf None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:38.716042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb87f0-2c45-4d72-84f4-c562cff7d08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:38.716654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb87f0-2c45-4d72-84f4-c562cff7d08a None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:38.716929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb87f0-2c45-4d72-84f4-c562cff7d08a None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:38.717196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb87f0-2c45-4d72-84f4-c562cff7d08a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:38.780664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb87f0-2c45-4d72-84f4-c562cff7d08a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:38.782102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb87f0-2c45-4d72-84f4-c562cff7d08a None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:38.876125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:38.885946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 645/1282] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 17:54:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 17:54:38.891807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6f12c-092c-4c84-bb86-30a4b6804808 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:38.952082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:38.989263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6f12c-092c-4c84-bb86-30a4b6804808 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:39.021885 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc996e75-d538-48d0-bec7-0dba01fc1f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:39.022494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc996e75-d538-48d0-bec7-0dba01fc1f9c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:39.022765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc996e75-d538-48d0-bec7-0dba01fc1f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:39.023027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc996e75-d538-48d0-bec7-0dba01fc1f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:39.032604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc996e75-d538-48d0-bec7-0dba01fc1f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:39.034120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc996e75-d538-48d0-bec7-0dba01fc1f9c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-dc996e75-d538-48d0-bec7-0dba01fc1f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 17:54:39.049143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:39.050301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 17:54:39.051425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 17:54:39.051425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 17:54:39.051425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:39.051425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 638/1283] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 17:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:39.055904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb87f0-2c45-4d72-84f4-c562cff7d08a None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:39.152277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb87f0-2c45-4d72-84f4-c562cff7d08a None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:39.185893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a62c8-711e-4a80-80ae-a997b4d79909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:39.186505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a62c8-711e-4a80-80ae-a997b4d79909 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:39.186776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a62c8-711e-4a80-80ae-a997b4d79909 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:39.187040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a62c8-711e-4a80-80ae-a997b4d79909 None admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:39.187669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a62c8-711e-4a80-80ae-a997b4d79909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:39.187970 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a62c8-711e-4a80-80ae-a997b4d79909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:39.189365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a62c8-711e-4a80-80ae-a997b4d79909 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:39.200228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 646/1284] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 17:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:39.206147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc996e75-d538-48d0-bec7-0dba01fc1f9c None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:39.305439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc996e75-d538-48d0-bec7-0dba01fc1f9c None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:39.318389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:39.338203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6504b6-b41d-46b5-a36d-363b7cd12e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:39.338811 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6504b6-b41d-46b5-a36d-363b7cd12e0a None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:39.339079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6504b6-b41d-46b5-a36d-363b7cd12e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:39.339337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6504b6-b41d-46b5-a36d-363b7cd12e0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:39.345766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6504b6-b41d-46b5-a36d-363b7cd12e0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:39.347166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6504b6-b41d-46b5-a36d-363b7cd12e0a None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application [None req-7f6504b6-b41d-46b5-a36d-363b7cd12e0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 17:54:39.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:39.359941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:39.361314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:54:39.361314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:54:39.361314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:54:39.361314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:54:39.361314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:54:39.361314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:54:39.361314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 17:54:39.361314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: ERROR keystone.server.flask.application Aug 08 17:54:39.361314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 639/1285] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 17:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:39.365853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a62c8-711e-4a80-80ae-a997b4d79909 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:39.415430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:39.463558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a62c8-711e-4a80-80ae-a997b4d79909 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:39.497867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5781b9b-db38-4911-b143-a2004f1ed5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:39.498461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5781b9b-db38-4911-b143-a2004f1ed5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:39.498731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5781b9b-db38-4911-b143-a2004f1ed5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:39.498997 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5781b9b-db38-4911-b143-a2004f1ed5e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:39.499679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5781b9b-db38-4911-b143-a2004f1ed5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:39.499977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5781b9b-db38-4911-b143-a2004f1ed5e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:39.501422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5781b9b-db38-4911-b143-a2004f1ed5e8 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:39.514046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 647/1286] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 17:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:39.521793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6504b6-b41d-46b5-a36d-363b7cd12e0a None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:39.617586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6504b6-b41d-46b5-a36d-363b7cd12e0a None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:39.651132 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b8ca1-9b51-494c-bc17-569dba61a6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:39.651716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b8ca1-9b51-494c-bc17-569dba61a6ac None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:39.651990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b8ca1-9b51-494c-bc17-569dba61a6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:39.652257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b8ca1-9b51-494c-bc17-569dba61a6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:39.653039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b8ca1-9b51-494c-bc17-569dba61a6ac None admin] RBAC: Adding query filter params (role.id=af39f6211a7244df8d429c0e561b9b2f, scope.project.id=35b6705a8d9647d3a9ea2b66510f8add, user.id=b81bdfeedcba4b0c93f13d14318b2151) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:39.653999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b8ca1-9b51-494c-bc17-569dba61a6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:39.654796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b8ca1-9b51-494c-bc17-569dba61a6ac None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:39.660341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 640/1287] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 17:54:39 2022] GET /identity/v3/role_assignments?user.id=b81bdfeedcba4b0c93f13d14318b2151&scope.project.id=35b6705a8d9647d3a9ea2b66510f8add&role.id=af39f6211a7244df8d429c0e561b9b2f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:41.661855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a9fa-faa1-48fe-a816-3b043e37c388 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:41.662384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a9fa-faa1-48fe-a816-3b043e37c388 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:41.662384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a9fa-faa1-48fe-a816-3b043e37c388 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:41.663760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 648/1288] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:41.669010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c6c6e-ef94-4ad2-8d39-865b4de80392 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:41.669426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c6c6e-ef94-4ad2-8d39-865b4de80392 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:41.669541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c6c6e-ef94-4ad2-8d39-865b4de80392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:41.687616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6a6c6c6e-ef94-4ad2-8d39-865b4de80392 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:41.688831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6a6c6c6e-ef94-4ad2-8d39-865b4de80392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:41.719216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:41.754972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 641/1289] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 17:54:41.777253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a9fa-faa1-48fe-a816-3b043e37c388 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:41.880275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a9fa-faa1-48fe-a816-3b043e37c388 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:41.890753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:41.916663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08370946-5ddb-4f41-9c8f-094282297a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:41.917222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08370946-5ddb-4f41-9c8f-094282297a38 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:41.917491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08370946-5ddb-4f41-9c8f-094282297a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:41.917753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08370946-5ddb-4f41-9c8f-094282297a38 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:41.918339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08370946-5ddb-4f41-9c8f-094282297a38 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:41.919730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08370946-5ddb-4f41-9c8f-094282297a38 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-08370946-5ddb-4f41-9c8f-094282297a38 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:41.929210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:41.930681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:41.931589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 649/1290] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:54:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:41.937243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c6c6e-ef94-4ad2-8d39-865b4de80392 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:42.048693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c6c6e-ef94-4ad2-8d39-865b4de80392 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:42.085881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d76db6-b2d1-4aab-a548-03e8ff97ca1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:42.086499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76db6-b2d1-4aab-a548-03e8ff97ca1c None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:42.086767 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76db6-b2d1-4aab-a548-03e8ff97ca1c None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:42.087031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76db6-b2d1-4aab-a548-03e8ff97ca1c None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:42.087665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d76db6-b2d1-4aab-a548-03e8ff97ca1c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:42.087959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d76db6-b2d1-4aab-a548-03e8ff97ca1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:42.089369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d76db6-b2d1-4aab-a548-03e8ff97ca1c None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:42.094856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 642/1291] 10.222.0.50 () {56 vars in 1116 bytes} [Mon Aug 8 17:54:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:42.100799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08370946-5ddb-4f41-9c8f-094282297a38 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:42.209095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08370946-5ddb-4f41-9c8f-094282297a38 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:42.245224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74792c2c-cd27-4f7c-b2ca-c59b0184ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:42.245825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74792c2c-cd27-4f7c-b2ca-c59b0184ea59 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:42.246094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74792c2c-cd27-4f7c-b2ca-c59b0184ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:42.246355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74792c2c-cd27-4f7c-b2ca-c59b0184ea59 None admin] PATH_INFO: `/v3/services` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:42.246983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74792c2c-cd27-4f7c-b2ca-c59b0184ea59 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:42.247278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74792c2c-cd27-4f7c-b2ca-c59b0184ea59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:42.249555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74792c2c-cd27-4f7c-b2ca-c59b0184ea59 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:42.256193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 650/1292] 10.222.0.50 () {56 vars in 1116 bytes} [Mon Aug 8 17:54:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:44.245170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d04c0-88ec-45f1-8e57-77e5d994f715 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:44.245716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d04c0-88ec-45f1-8e57-77e5d994f715 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:44.245716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d04c0-88ec-45f1-8e57-77e5d994f715 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:44.247014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 643/1293] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:44.252058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b65df6-77c7-438b-aa75-654515cfd2ca None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:44.252451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b65df6-77c7-438b-aa75-654515cfd2ca None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:44.252636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b65df6-77c7-438b-aa75-654515cfd2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:44.270775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-05b65df6-77c7-438b-aa75-654515cfd2ca None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:44.271925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-05b65df6-77c7-438b-aa75-654515cfd2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:44.301330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:44.336933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 651/1294] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:54:44.359379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d04c0-88ec-45f1-8e57-77e5d994f715 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:44.389413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:44.458425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d04c0-88ec-45f1-8e57-77e5d994f715 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:44.490905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a5009-2ef0-4374-9ccd-94858dc56f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:44.491482 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a5009-2ef0-4374-9ccd-94858dc56f77 None admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:44.491747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a5009-2ef0-4374-9ccd-94858dc56f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:44.492003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a5009-2ef0-4374-9ccd-94858dc56f77 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:44.492666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a5009-2ef0-4374-9ccd-94858dc56f77 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:44.494054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a5009-2ef0-4374-9ccd-94858dc56f77 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:44.612145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 644/1295] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 17:54:44 2022] POST /identity/v3/services => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:54:46.598283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f90d21-445a-47f0-b795-7e88a030c43b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:46.598839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f90d21-445a-47f0-b795-7e88a030c43b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:46.598839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f90d21-445a-47f0-b795-7e88a030c43b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:46.600163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 652/1296] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:46.607647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766604bb-10ed-45e3-93e4-05546b6b03a8 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:46.608117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766604bb-10ed-45e3-93e4-05546b6b03a8 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:46.610399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766604bb-10ed-45e3-93e4-05546b6b03a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:46.630240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-766604bb-10ed-45e3-93e4-05546b6b03a8 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:46.631418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-766604bb-10ed-45e3-93e4-05546b6b03a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:46.701855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 645/1297] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:54:46.724955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f90d21-445a-47f0-b795-7e88a030c43b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:46.826558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f90d21-445a-47f0-b795-7e88a030c43b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:46.859114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466889a-1a50-4f4e-9ac8-72e1c2927a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:46.859667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466889a-1a50-4f4e-9ac8-72e1c2927a4b None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:46.859942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466889a-1a50-4f4e-9ac8-72e1c2927a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:46.860207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466889a-1a50-4f4e-9ac8-72e1c2927a4b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:46.860808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466889a-1a50-4f4e-9ac8-72e1c2927a4b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:46.862262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466889a-1a50-4f4e-9ac8-72e1c2927a4b None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-6466889a-1a50-4f4e-9ac8-72e1c2927a4b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:46.871149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:46.872337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:46.873263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 653/1298] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:54:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:46.878663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766604bb-10ed-45e3-93e4-05546b6b03a8 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:46.897016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:46.975598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766604bb-10ed-45e3-93e4-05546b6b03a8 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:47.008858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826a05a-d991-4cb2-8b1f-37b17de38103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:47.009512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826a05a-d991-4cb2-8b1f-37b17de38103 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:47.009762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826a05a-d991-4cb2-8b1f-37b17de38103 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:47.010021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826a05a-d991-4cb2-8b1f-37b17de38103 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:47.010636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826a05a-d991-4cb2-8b1f-37b17de38103 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:47.010922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826a05a-d991-4cb2-8b1f-37b17de38103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:47.012262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826a05a-d991-4cb2-8b1f-37b17de38103 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:47.017221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 646/1299] 10.222.0.50 () {56 vars in 1116 bytes} [Mon Aug 8 17:54:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:47.023268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466889a-1a50-4f4e-9ac8-72e1c2927a4b None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:47.054796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:47.130682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466889a-1a50-4f4e-9ac8-72e1c2927a4b None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:47.163556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853e94b-6e3b-4281-8545-144ea94a109e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:47.164183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853e94b-6e3b-4281-8545-144ea94a109e None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:47.164461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853e94b-6e3b-4281-8545-144ea94a109e None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:47.164763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853e94b-6e3b-4281-8545-144ea94a109e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:47.165454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853e94b-6e3b-4281-8545-144ea94a109e None admin] RBAC: Adding query filter params (interface=public, service_id=df68d2a6c71a46d7956ff97261ee22e7, region_id=RegionOne) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:47.165758 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853e94b-6e3b-4281-8545-144ea94a109e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:47.167129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853e94b-6e3b-4281-8545-144ea94a109e None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:47.172277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 654/1300] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 17:54:47 2022] GET /identity/v3/endpoints?service_id=df68d2a6c71a46d7956ff97261ee22e7&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:54:47.178664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826a05a-d991-4cb2-8b1f-37b17de38103 None admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:47.260700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:47.278197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826a05a-d991-4cb2-8b1f-37b17de38103 None admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:47.311941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a2d43-195c-4141-810b-b3616912f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:47.312555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a2d43-195c-4141-810b-b3616912f556 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:47.312839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a2d43-195c-4141-810b-b3616912f556 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:47.313089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a2d43-195c-4141-810b-b3616912f556 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:47.313681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a2d43-195c-4141-810b-b3616912f556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:47.315032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a2d43-195c-4141-810b-b3616912f556 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:47.321632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 647/1301] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 17:54:47 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:54:49.299681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb060bb-9751-4495-8e92-aaf5c2c07c03 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:49.300199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb060bb-9751-4495-8e92-aaf5c2c07c03 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:49.300199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb060bb-9751-4495-8e92-aaf5c2c07c03 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:49.301683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 655/1302] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 17:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:54:49.307252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a930b63-081b-4cac-88ae-49e9831b6129 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:49.307647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a930b63-081b-4cac-88ae-49e9831b6129 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:49.307788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a930b63-081b-4cac-88ae-49e9831b6129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:49.326338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5a930b63-081b-4cac-88ae-49e9831b6129 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:54:49.327515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5a930b63-081b-4cac-88ae-49e9831b6129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:54:49.393375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 648/1303] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 17:54:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 17:54:49.416278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb060bb-9751-4495-8e92-aaf5c2c07c03 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:49.511748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:49.518381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb060bb-9751-4495-8e92-aaf5c2c07c03 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:49.551749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2b4f0-76dd-49a2-b4ca-400b0705ecf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:49.552284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2b4f0-76dd-49a2-b4ca-400b0705ecf6 None admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:49.552586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2b4f0-76dd-49a2-b4ca-400b0705ecf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:49.552855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2b4f0-76dd-49a2-b4ca-400b0705ecf6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:49.553428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2b4f0-76dd-49a2-b4ca-400b0705ecf6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:49.554779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2b4f0-76dd-49a2-b4ca-400b0705ecf6 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application [None req-4ce2b4f0-76dd-49a2-b4ca-400b0705ecf6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:54:49.564146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:54:49.569589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application with Lock( Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: ERROR keystone.server.flask.application Aug 08 17:54:49.570449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 656/1304] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 17:54:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 17:54:49.572169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a930b63-081b-4cac-88ae-49e9831b6129 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:49.646049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:49.667087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a930b63-081b-4cac-88ae-49e9831b6129 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:49.699723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788940a-44be-4d90-83d6-b7d758763a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:49.700306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788940a-44be-4d90-83d6-b7d758763a42 None admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:49.700607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788940a-44be-4d90-83d6-b7d758763a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:49.701127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788940a-44be-4d90-83d6-b7d758763a42 None admin] PATH_INFO: `/v3/services` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:49.701738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788940a-44be-4d90-83d6-b7d758763a42 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:54:49.702027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788940a-44be-4d90-83d6-b7d758763a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:49.703367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788940a-44be-4d90-83d6-b7d758763a42 None admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:49.711404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 649/1305] 10.222.0.50 () {56 vars in 1116 bytes} [Mon Aug 8 17:54:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:54:49.714322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2b4f0-76dd-49a2-b4ca-400b0705ecf6 None admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:54:49.812604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2b4f0-76dd-49a2-b4ca-400b0705ecf6 None admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:54:49.845459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e75171-17b4-4cae-99ab-97dbda830390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:54:49.846040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e75171-17b4-4cae-99ab-97dbda830390 None admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:54:49.846342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e75171-17b4-4cae-99ab-97dbda830390 None admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:54:49.846604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e75171-17b4-4cae-99ab-97dbda830390 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:54:49.847233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e75171-17b4-4cae-99ab-97dbda830390 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:54:49.848602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e75171-17b4-4cae-99ab-97dbda830390 None admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:54:49.932289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 657/1306] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 17:54:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:54:59.728264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=30 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:54:59.927663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:01.333347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe3a7c-dc41-4e6d-b1a3-1bfd04cebe40 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:01.334127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe3a7c-dc41-4e6d-b1a3-1bfd04cebe40 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:01.334127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe3a7c-dc41-4e6d-b1a3-1bfd04cebe40 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:01.340380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d4ccc-0417-43c2-b103-cff43d247a0b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:01.341107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d4ccc-0417-43c2-b103-cff43d247a0b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:01.341107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d4ccc-0417-43c2-b103-cff43d247a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:01.345659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 650/1307] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:01.414483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-595d4ccc-0417-43c2-b103-cff43d247a0b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:01.423856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-595d4ccc-0417-43c2-b103-cff43d247a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:01.540722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 658/1308] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:01.574846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fe72a-c62b-4ef9-870e-9cd46c348574 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:01.575114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fe72a-c62b-4ef9-870e-9cd46c348574 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:01.575453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fe72a-c62b-4ef9-870e-9cd46c348574 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:01.577551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 651/1309] 10.222.0.50 () {54 vars in 965 bytes} [Mon Aug 8 18:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:01.584218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793aa6-f74d-4df7-b0fa-e09b906b4325 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:01.584467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793aa6-f74d-4df7-b0fa-e09b906b4325 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:01.584899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793aa6-f74d-4df7-b0fa-e09b906b4325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:01.667255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-23793aa6-f74d-4df7-b0fa-e09b906b4325 None None] MFA Rules not processed for user `95f02466042946ee96e37c5adc27b01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:01.668358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-23793aa6-f74d-4df7-b0fa-e09b906b4325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:01.689428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:01.755678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fe72a-c62b-4ef9-870e-9cd46c348574 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:01.757627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c91fe72a-c62b-4ef9-870e-9cd46c348574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:01.767260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 659/1310] 10.222.0.50 () {58 vars in 1062 bytes} [Mon Aug 8 18:01:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:01:01.890973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fe72a-c62b-4ef9-870e-9cd46c348574 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:01.971932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8532bf8-d7b5-4da6-8f80-12eadbc339c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:01.972657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8532bf8-d7b5-4da6-8f80-12eadbc339c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:01.973119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8532bf8-d7b5-4da6-8f80-12eadbc339c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:01.973388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8532bf8-d7b5-4da6-8f80-12eadbc339c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:01.974975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c8532bf8-d7b5-4da6-8f80-12eadbc339c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:02.115479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:02.227253 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8532bf8-d7b5-4da6-8f80-12eadbc339c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:02.229095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8532bf8-d7b5-4da6-8f80-12eadbc339c2 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:02.252809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 652/1311] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:04.626180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be8e79-b3fb-4e95-bf41-dd4530f56b0d None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:04.626707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be8e79-b3fb-4e95-bf41-dd4530f56b0d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:04.626707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be8e79-b3fb-4e95-bf41-dd4530f56b0d None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:04.628211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 660/1312] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:04.633254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6f0be-dbe1-4273-9820-a0c27689f45a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:04.633652 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6f0be-dbe1-4273-9820-a0c27689f45a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:04.633793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6f0be-dbe1-4273-9820-a0c27689f45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:04.715907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-28d6f0be-dbe1-4273-9820-a0c27689f45a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:04.717685 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-28d6f0be-dbe1-4273-9820-a0c27689f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:04.816456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 653/1313] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:04.869758 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55d59c-7e00-4e0e-a95b-eb193973614b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:04.870037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55d59c-7e00-4e0e-a95b-eb193973614b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:04.870307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55d59c-7e00-4e0e-a95b-eb193973614b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:04.871660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 661/1314] 10.222.0.50 () {54 vars in 965 bytes} [Mon Aug 8 18:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:04.877440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3f1b-7cbc-48f5-93e2-c6ae2ef7dd7e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:04.877724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3f1b-7cbc-48f5-93e2-c6ae2ef7dd7e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:04.877993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3f1b-7cbc-48f5-93e2-c6ae2ef7dd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:04.890016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:04.956682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-576e3f1b-7cbc-48f5-93e2-c6ae2ef7dd7e None None] MFA Rules not processed for user `95f02466042946ee96e37c5adc27b01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:04.957900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-576e3f1b-7cbc-48f5-93e2-c6ae2ef7dd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:05.036306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 654/1315] 10.222.0.50 () {58 vars in 1062 bytes} [Mon Aug 8 18:01:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:05.045429 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55d59c-7e00-4e0e-a95b-eb193973614b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:05.046965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6c55d59c-7e00-4e0e-a95b-eb193973614b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:05.151845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55d59c-7e00-4e0e-a95b-eb193973614b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:05.230634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:05.233511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0ebbd-c971-481c-b743-a9d90e4e216c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:05.234119 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ebbd-c971-481c-b743-a9d90e4e216c service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:05.234394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ebbd-c971-481c-b743-a9d90e4e216c service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:05.234660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ebbd-c971-481c-b743-a9d90e4e216c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:05.236311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0db0ebbd-c971-481c-b743-a9d90e4e216c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:05.351236 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0ebbd-c971-481c-b743-a9d90e4e216c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:05.352706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0ebbd-c971-481c-b743-a9d90e4e216c service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:05.371274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 662/1316] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:08.107996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb82f-532b-4df7-8564-213304d7222d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:08.108519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb82f-532b-4df7-8564-213304d7222d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:08.108519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb82f-532b-4df7-8564-213304d7222d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:08.109821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 655/1317] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:08.114845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f07da-d52f-4980-853b-b2519ef600a5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:08.115265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f07da-d52f-4980-853b-b2519ef600a5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:08.115359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f07da-d52f-4980-853b-b2519ef600a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:08.179763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8c5f07da-d52f-4980-853b-b2519ef600a5 None None] MFA Rules not processed for user `a5e74655bc0d48558528bf252f846eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:08.181033 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8c5f07da-d52f-4980-853b-b2519ef600a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:08.231622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:08.274431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 663/1318] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:08.314957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb82f-532b-4df7-8564-213304d7222d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:08.316475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-52beb82f-532b-4df7-8564-213304d7222d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:08.415892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb82f-532b-4df7-8564-213304d7222d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:08.418583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:08.492295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c59e5-7352-45c1-a566-deddd1aef3a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:08.492913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c59e5-7352-45c1-a566-deddd1aef3a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:08.493187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c59e5-7352-45c1-a566-deddd1aef3a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:08.493448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c59e5-7352-45c1-a566-deddd1aef3a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:08.495017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-560c59e5-7352-45c1-a566-deddd1aef3a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:08.601210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c59e5-7352-45c1-a566-deddd1aef3a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:08.602645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c59e5-7352-45c1-a566-deddd1aef3a8 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:08.621299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 656/1319] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:11.412938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a17ae-76bd-4e5a-824b-25371495b13c None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:11.413486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a17ae-76bd-4e5a-824b-25371495b13c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:11.413486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a17ae-76bd-4e5a-824b-25371495b13c None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:11.414801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 664/1320] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:11.419820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398477c4-a6ed-46cf-a302-30644e2536f0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:11.420246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398477c4-a6ed-46cf-a302-30644e2536f0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:11.420364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398477c4-a6ed-46cf-a302-30644e2536f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:11.460388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:11.484963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-398477c4-a6ed-46cf-a302-30644e2536f0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:11.486179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-398477c4-a6ed-46cf-a302-30644e2536f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:11.581831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 657/1321] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:01:11.620112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a17ae-76bd-4e5a-824b-25371495b13c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:11.621743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-951a17ae-76bd-4e5a-824b-25371495b13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:11.728936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a17ae-76bd-4e5a-824b-25371495b13c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:11.801367 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41206f-581b-4d12-ad3f-eec616682e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:11.802114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41206f-581b-4d12-ad3f-eec616682e36 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:11.802236 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41206f-581b-4d12-ad3f-eec616682e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:11.802509 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41206f-581b-4d12-ad3f-eec616682e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:11.804190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cb41206f-581b-4d12-ad3f-eec616682e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:11.812301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:11.912855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41206f-581b-4d12-ad3f-eec616682e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:11.914297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41206f-581b-4d12-ad3f-eec616682e36 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:11.931955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 665/1322] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:15.114582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588eaa-306c-4c54-aefb-0c3476c56d6b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:15.115105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588eaa-306c-4c54-aefb-0c3476c56d6b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:15.115105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588eaa-306c-4c54-aefb-0c3476c56d6b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:15.116383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 658/1323] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:15.121518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd7956-6372-4ca6-99e4-066c4c754707 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:15.121932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd7956-6372-4ca6-99e4-066c4c754707 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:15.122187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd7956-6372-4ca6-99e4-066c4c754707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:15.190906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-44dd7956-6372-4ca6-99e4-066c4c754707 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:15.192124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-44dd7956-6372-4ca6-99e4-066c4c754707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:15.250128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:15.289101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 666/1324] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:15.324190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35588eaa-306c-4c54-aefb-0c3476c56d6b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:15.326237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-35588eaa-306c-4c54-aefb-0c3476c56d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:15.438323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35588eaa-306c-4c54-aefb-0c3476c56d6b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:15.452735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:15.520427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66caa9b1-3a22-4bd7-a267-73e6575f6325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:15.521070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66caa9b1-3a22-4bd7-a267-73e6575f6325 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:15.521342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66caa9b1-3a22-4bd7-a267-73e6575f6325 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:15.521606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66caa9b1-3a22-4bd7-a267-73e6575f6325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:15.523264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-66caa9b1-3a22-4bd7-a267-73e6575f6325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:15.635136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66caa9b1-3a22-4bd7-a267-73e6575f6325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:15.636590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66caa9b1-3a22-4bd7-a267-73e6575f6325 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:15.655729 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 659/1325] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:01:17.702292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c3dad-cd0c-4d28-a4b0-bfc16ed999e6 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:17.702881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c3dad-cd0c-4d28-a4b0-bfc16ed999e6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:17.702881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c3dad-cd0c-4d28-a4b0-bfc16ed999e6 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:17.704425 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 667/1326] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:17.709507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c54fb-1d3b-4790-bdcc-edebd67a195f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:17.709940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c54fb-1d3b-4790-bdcc-edebd67a195f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:17.710174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c54fb-1d3b-4790-bdcc-edebd67a195f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:17.757070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:17.775285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d83c54fb-1d3b-4790-bdcc-edebd67a195f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:17.776484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d83c54fb-1d3b-4790-bdcc-edebd67a195f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:17.874504 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 660/1327] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:01:17.909761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c3dad-cd0c-4d28-a4b0-bfc16ed999e6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:17.911291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-537c3dad-cd0c-4d28-a4b0-bfc16ed999e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:18.020982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c3dad-cd0c-4d28-a4b0-bfc16ed999e6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:18.092481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b7f3c-a24e-4dfc-893f-003f517bcd1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:18.093035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b7f3c-a24e-4dfc-893f-003f517bcd1b service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:18.093308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b7f3c-a24e-4dfc-893f-003f517bcd1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:18.093573 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b7f3c-a24e-4dfc-893f-003f517bcd1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:18.095149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0e8b7f3c-a24e-4dfc-893f-003f517bcd1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:18.111791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:18.202777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b7f3c-a24e-4dfc-893f-003f517bcd1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:18.204461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b7f3c-a24e-4dfc-893f-003f517bcd1b service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:18.221865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 668/1328] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:20.272601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96267a6d-a282-4d6b-bb6b-dc17f93f343d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:20.273164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96267a6d-a282-4d6b-bb6b-dc17f93f343d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:20.273164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96267a6d-a282-4d6b-bb6b-dc17f93f343d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:20.274533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 661/1329] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:20.280263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6923b0-1bf4-44d1-8ee9-1e0e93a6133a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:20.280698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6923b0-1bf4-44d1-8ee9-1e0e93a6133a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:20.280983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6923b0-1bf4-44d1-8ee9-1e0e93a6133a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:20.349315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1f6923b0-1bf4-44d1-8ee9-1e0e93a6133a None None] MFA Rules not processed for user `a5e74655bc0d48558528bf252f846eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:20.350564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1f6923b0-1bf4-44d1-8ee9-1e0e93a6133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:20.418175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:20.451299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 669/1330] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:20.488733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96267a6d-a282-4d6b-bb6b-dc17f93f343d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:20.490252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-96267a6d-a282-4d6b-bb6b-dc17f93f343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:20.593520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96267a6d-a282-4d6b-bb6b-dc17f93f343d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:20.614237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:20.668694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa19f2-2c14-41bb-a501-a9fc9d8f097b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:20.669323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa19f2-2c14-41bb-a501-a9fc9d8f097b service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:20.669590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa19f2-2c14-41bb-a501-a9fc9d8f097b service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:20.669851 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa19f2-2c14-41bb-a501-a9fc9d8f097b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:20.671432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-acaa19f2-2c14-41bb-a501-a9fc9d8f097b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:20.783244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa19f2-2c14-41bb-a501-a9fc9d8f097b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:20.784706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa19f2-2c14-41bb-a501-a9fc9d8f097b service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:20.805058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 662/1331] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:23.340264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073381e-f2da-4780-8a30-ee8cd1bdaa5e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:23.340892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073381e-f2da-4780-8a30-ee8cd1bdaa5e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:23.340892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073381e-f2da-4780-8a30-ee8cd1bdaa5e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:23.342733 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 670/1332] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:23.347960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f71e2-320c-4839-9b1c-42eb37ea9c2a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:23.348387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f71e2-320c-4839-9b1c-42eb37ea9c2a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:23.348490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f71e2-320c-4839-9b1c-42eb37ea9c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:23.411214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:23.413865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9f3f71e2-320c-4839-9b1c-42eb37ea9c2a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:23.415064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9f3f71e2-320c-4839-9b1c-42eb37ea9c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:23.508881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 663/1333] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:23.545810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073381e-f2da-4780-8a30-ee8cd1bdaa5e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:23.547431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9073381e-f2da-4780-8a30-ee8cd1bdaa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:23.653254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073381e-f2da-4780-8a30-ee8cd1bdaa5e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:23.727381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d6624-62cd-4b29-9d09-302dda47ca1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:23.727981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d6624-62cd-4b29-9d09-302dda47ca1a service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:23.728253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d6624-62cd-4b29-9d09-302dda47ca1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:23.728529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d6624-62cd-4b29-9d09-302dda47ca1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:23.730156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ae9d6624-62cd-4b29-9d09-302dda47ca1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:23.756784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:23.838544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d6624-62cd-4b29-9d09-302dda47ca1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:23.839974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d6624-62cd-4b29-9d09-302dda47ca1a service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:23.865116 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 671/1334] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:25.892886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d937de-3fa9-4d6a-91cf-7bea1e17d4f9 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:25.893408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d937de-3fa9-4d6a-91cf-7bea1e17d4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:25.893408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d937de-3fa9-4d6a-91cf-7bea1e17d4f9 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:25.894759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 664/1335] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:25.899944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6a75f-8c73-4ac3-a836-1fa6e97d9441 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:25.900412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6a75f-8c73-4ac3-a836-1fa6e97d9441 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:25.900721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6a75f-8c73-4ac3-a836-1fa6e97d9441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:25.965337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-97c6a75f-8c73-4ac3-a836-1fa6e97d9441 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:25.967037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-97c6a75f-8c73-4ac3-a836-1fa6e97d9441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:26.039529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:26.063112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 672/1336] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:26.103070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d937de-3fa9-4d6a-91cf-7bea1e17d4f9 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:26.104660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-80d937de-3fa9-4d6a-91cf-7bea1e17d4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:26.205808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d937de-3fa9-4d6a-91cf-7bea1e17d4f9 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:26.234054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:26.282128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9433dea-2d84-440c-90fb-57583f70907d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:26.282715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9433dea-2d84-440c-90fb-57583f70907d service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:26.282981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9433dea-2d84-440c-90fb-57583f70907d service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:26.283238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9433dea-2d84-440c-90fb-57583f70907d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:26.284877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b9433dea-2d84-440c-90fb-57583f70907d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:26.390677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9433dea-2d84-440c-90fb-57583f70907d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:26.392006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9433dea-2d84-440c-90fb-57583f70907d service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:26.411832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 665/1337] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:29.204393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eac75-59a2-449d-bc7a-e1a565a771de None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:29.204942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eac75-59a2-449d-bc7a-e1a565a771de None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:29.204942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eac75-59a2-449d-bc7a-e1a565a771de None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:29.206256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 673/1338] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:29.211815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be16a30-7b8b-43af-a4a0-57aadae8b1a9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:29.212280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be16a30-7b8b-43af-a4a0-57aadae8b1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:29.212377 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be16a30-7b8b-43af-a4a0-57aadae8b1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:29.276683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3be16a30-7b8b-43af-a4a0-57aadae8b1a9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:29.277883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3be16a30-7b8b-43af-a4a0-57aadae8b1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:29.286365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:29.372123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 666/1339] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:29.408460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173eac75-59a2-449d-bc7a-e1a565a771de None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:29.410041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-173eac75-59a2-449d-bc7a-e1a565a771de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:29.510283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173eac75-59a2-449d-bc7a-e1a565a771de None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:29.583469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ca013-589e-4af7-b5ce-9fcb374a7182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:29.584110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca013-589e-4af7-b5ce-9fcb374a7182 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:29.584378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca013-589e-4af7-b5ce-9fcb374a7182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:29.584674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca013-589e-4af7-b5ce-9fcb374a7182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:29.586240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9d9ca013-589e-4af7-b5ce-9fcb374a7182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:29.621479 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:29.692355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ca013-589e-4af7-b5ce-9fcb374a7182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:29.693874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ca013-589e-4af7-b5ce-9fcb374a7182 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:29.712227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 674/1340] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:34.042821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c57cd-7d7f-4d76-b152-5b78653bf801 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:34.043282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c57cd-7d7f-4d76-b152-5b78653bf801 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:34.043376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c57cd-7d7f-4d76-b152-5b78653bf801 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:34.044896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 667/1341] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:34.049973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafd1c-6f31-49bd-9dd7-0b6e07c38ea8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:34.050394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafd1c-6f31-49bd-9dd7-0b6e07c38ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:34.050491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafd1c-6f31-49bd-9dd7-0b6e07c38ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:34.117707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3baafd1c-6f31-49bd-9dd7-0b6e07c38ea8 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:34.118900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3baafd1c-6f31-49bd-9dd7-0b6e07c38ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:34.204181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:34.215268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 675/1342] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:34.254374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c57cd-7d7f-4d76-b152-5b78653bf801 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:34.255899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-059c57cd-7d7f-4d76-b152-5b78653bf801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:34.359703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c57cd-7d7f-4d76-b152-5b78653bf801 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:34.398278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:34.433611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e80727-98c7-4168-8afd-a55ca3dac3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:34.434807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e80727-98c7-4168-8afd-a55ca3dac3dc service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:34.435144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e80727-98c7-4168-8afd-a55ca3dac3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:34.435414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e80727-98c7-4168-8afd-a55ca3dac3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:34.437029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-24e80727-98c7-4168-8afd-a55ca3dac3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:34.545015 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e80727-98c7-4168-8afd-a55ca3dac3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:34.546451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e80727-98c7-4168-8afd-a55ca3dac3dc service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:34.566344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 668/1343] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:01:37.530683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f8761-6a02-4903-b402-b94f31b8aad5 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:37.531189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f8761-6a02-4903-b402-b94f31b8aad5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:37.531189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f8761-6a02-4903-b402-b94f31b8aad5 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:37.532551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 676/1344] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:37.537775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e8be3-1776-4029-9312-6d3c5aaa40e3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:37.538234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e8be3-1776-4029-9312-6d3c5aaa40e3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:37.538334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e8be3-1776-4029-9312-6d3c5aaa40e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:37.605575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-986e8be3-1776-4029-9312-6d3c5aaa40e3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:37.607356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-986e8be3-1776-4029-9312-6d3c5aaa40e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:37.628786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:37.713395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 669/1345] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:37.751722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f8761-6a02-4903-b402-b94f31b8aad5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:37.753449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-890f8761-6a02-4903-b402-b94f31b8aad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:37.863785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f8761-6a02-4903-b402-b94f31b8aad5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:37.939130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41706b0-d4c8-4e67-b922-d2f48779ca7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:37.939734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41706b0-d4c8-4e67-b922-d2f48779ca7f service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:37.940010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41706b0-d4c8-4e67-b922-d2f48779ca7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:37.940278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41706b0-d4c8-4e67-b922-d2f48779ca7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:37.941915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f41706b0-d4c8-4e67-b922-d2f48779ca7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:37.986045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:38.056100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41706b0-d4c8-4e67-b922-d2f48779ca7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:38.057521 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41706b0-d4c8-4e67-b922-d2f48779ca7f service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:38.077433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 677/1346] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:42.229502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d052fdd-5423-47f0-89c4-34bd248c353c None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:42.230002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d052fdd-5423-47f0-89c4-34bd248c353c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:42.230100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d052fdd-5423-47f0-89c4-34bd248c353c None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:42.231413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 670/1347] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:42.236591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bcee7-1272-4dd6-9dfb-0fcc50eaade6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:42.237014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bcee7-1272-4dd6-9dfb-0fcc50eaade6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:42.237134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bcee7-1272-4dd6-9dfb-0fcc50eaade6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:42.311175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3e6bcee7-1272-4dd6-9dfb-0fcc50eaade6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:42.312403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3e6bcee7-1272-4dd6-9dfb-0fcc50eaade6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:42.411835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:42.414628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 678/1348] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:01:42.453907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d052fdd-5423-47f0-89c4-34bd248c353c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:42.455425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6d052fdd-5423-47f0-89c4-34bd248c353c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:42.564018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d052fdd-5423-47f0-89c4-34bd248c353c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:42.614613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:42.642483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d8012-9822-466f-911d-4fbc1f707ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:42.643003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8012-9822-466f-911d-4fbc1f707ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:42.643274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8012-9822-466f-911d-4fbc1f707ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:42.643536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8012-9822-466f-911d-4fbc1f707ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:42.645167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5d0d8012-9822-466f-911d-4fbc1f707ff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:42.757785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d8012-9822-466f-911d-4fbc1f707ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:42.759218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d8012-9822-466f-911d-4fbc1f707ff2 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:42.778404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 671/1349] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:01:47.826955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a307d-daf1-4f45-9f5b-c5fd56a48cef None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:47.827488 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a307d-daf1-4f45-9f5b-c5fd56a48cef None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:47.827488 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a307d-daf1-4f45-9f5b-c5fd56a48cef None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:47.829710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 679/1350] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:47.837439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29258414-fbb7-4191-a15e-7de7eb7b8188 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:47.838375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29258414-fbb7-4191-a15e-7de7eb7b8188 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:47.838864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29258414-fbb7-4191-a15e-7de7eb7b8188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:47.940344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-29258414-fbb7-4191-a15e-7de7eb7b8188 None None] MFA Rules not processed for user `b81bdfeedcba4b0c93f13d14318b2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:47.941575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-29258414-fbb7-4191-a15e-7de7eb7b8188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:47.963105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:48.061624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 672/1351] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:01:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:01:52.143454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f8fb1-a5c5-4e08-b03f-3538e403c504 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:52.143982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f8fb1-a5c5-4e08-b03f-3538e403c504 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:52.144422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f8fb1-a5c5-4e08-b03f-3538e403c504 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:52.145630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 680/1352] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:52.150765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf420-e45d-4a3a-901d-62cd97849b33 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:52.151213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf420-e45d-4a3a-901d-62cd97849b33 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:52.151552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf420-e45d-4a3a-901d-62cd97849b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:52.231837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f9edf420-e45d-4a3a-901d-62cd97849b33 None None] MFA Rules not processed for user `b81bdfeedcba4b0c93f13d14318b2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:52.233221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f9edf420-e45d-4a3a-901d-62cd97849b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:52.305947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:52.344194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 673/1353] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:01:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:53.679853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8bba-e1a2-4aae-bc2e-b28973c66b27 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:53.680394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8bba-e1a2-4aae-bc2e-b28973c66b27 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:53.680394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8bba-e1a2-4aae-bc2e-b28973c66b27 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:53.681767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 681/1354] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:53.687470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df0ccf-d702-4510-9933-3ccd24e45610 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:53.687868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df0ccf-d702-4510-9933-3ccd24e45610 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:53.688176 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df0ccf-d702-4510-9933-3ccd24e45610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:53.767755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c7df0ccf-d702-4510-9933-3ccd24e45610 None None] MFA Rules not processed for user `b81bdfeedcba4b0c93f13d14318b2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:53.769362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c7df0ccf-d702-4510-9933-3ccd24e45610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:53.882437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 674/1355] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:01:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:01:57.359979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff584365-35f0-49d3-a210-b6c2f2aa2057 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:57.360479 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff584365-35f0-49d3-a210-b6c2f2aa2057 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:57.360479 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff584365-35f0-49d3-a210-b6c2f2aa2057 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:57.361784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 682/1356] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:57.367108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874d1e8-64fe-44f3-8131-a10ea59035d0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:57.367858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874d1e8-64fe-44f3-8131-a10ea59035d0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:57.368125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874d1e8-64fe-44f3-8131-a10ea59035d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:57.411284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:57.443041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8874d1e8-64fe-44f3-8131-a10ea59035d0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:57.444950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8874d1e8-64fe-44f3-8131-a10ea59035d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:57.546343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 675/1357] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:57.603181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e786e-f54e-437e-af7e-42a82ddba89f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:57.603831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e786e-f54e-437e-af7e-42a82ddba89f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:57.604321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e786e-f54e-437e-af7e-42a82ddba89f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:57.606501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 683/1358] 10.222.0.50 () {54 vars in 962 bytes} [Mon Aug 8 18:01:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:01:57.615184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f6b74-b665-4ddc-bbd7-b65c10ae1bef None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:57.615471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f6b74-b665-4ddc-bbd7-b65c10ae1bef None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:57.620253 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f6b74-b665-4ddc-bbd7-b65c10ae1bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:57.701505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-529f6b74-b665-4ddc-bbd7-b65c10ae1bef None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:01:57.703121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-529f6b74-b665-4ddc-bbd7-b65c10ae1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:57.755693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:57.840809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 676/1359] 10.222.0.50 () {58 vars in 1059 bytes} [Mon Aug 8 18:01:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:01:57.844592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e786e-f54e-437e-af7e-42a82ddba89f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:01:57.846314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-280e786e-f54e-437e-af7e-42a82ddba89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:58.001863 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e786e-f54e-437e-af7e-42a82ddba89f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:01:58.079632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:01:58.122731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956429ac-9431-4355-9541-b0a4ce7207a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:01:58.123412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956429ac-9431-4355-9541-b0a4ce7207a3 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:01:58.123756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956429ac-9431-4355-9541-b0a4ce7207a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:01:58.124088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956429ac-9431-4355-9541-b0a4ce7207a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:01:58.125808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-956429ac-9431-4355-9541-b0a4ce7207a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:01:58.226341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956429ac-9431-4355-9541-b0a4ce7207a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:01:58.228220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956429ac-9431-4355-9541-b0a4ce7207a3 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:01:58.254589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 684/1360] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:00.467416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447a80b-f4cb-42d5-9681-7795d3d8b7ab None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:00.467941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447a80b-f4cb-42d5-9681-7795d3d8b7ab None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:00.467941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447a80b-f4cb-42d5-9681-7795d3d8b7ab None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:00.469467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 677/1361] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:00.474444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fbabd-61bc-41fb-9918-a5fbc032c6dd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:00.474837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fbabd-61bc-41fb-9918-a5fbc032c6dd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:00.475013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fbabd-61bc-41fb-9918-a5fbc032c6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:00.544021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-077fbabd-61bc-41fb-9918-a5fbc032c6dd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:00.545294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-077fbabd-61bc-41fb-9918-a5fbc032c6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:00.583916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:00.645626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 685/1362] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:00.693251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447a80b-f4cb-42d5-9681-7795d3d8b7ab None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:00.694829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6447a80b-f4cb-42d5-9681-7795d3d8b7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:00.876947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447a80b-f4cb-42d5-9681-7795d3d8b7ab None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:00.886822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:01.031884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512d0b3-e821-4a52-8943-d39054d3647c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:01.032825 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512d0b3-e821-4a52-8943-d39054d3647c service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:01.033204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512d0b3-e821-4a52-8943-d39054d3647c service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:01.033576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512d0b3-e821-4a52-8943-d39054d3647c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:01.035302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3512d0b3-e821-4a52-8943-d39054d3647c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:01.132353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512d0b3-e821-4a52-8943-d39054d3647c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:01.134107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512d0b3-e821-4a52-8943-d39054d3647c service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:01.153628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 678/1363] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:03.691543 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308958f7-7538-4740-8c2a-08a19d55c0db None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:03.692089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308958f7-7538-4740-8c2a-08a19d55c0db None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:03.692089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308958f7-7538-4740-8c2a-08a19d55c0db None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:03.693434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 686/1364] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:03.700148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78805af-8b4f-4ea8-adec-78e9e0d2c20f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:03.700833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78805af-8b4f-4ea8-adec-78e9e0d2c20f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:03.700833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78805af-8b4f-4ea8-adec-78e9e0d2c20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:03.730064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:03.792657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b78805af-8b4f-4ea8-adec-78e9e0d2c20f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:03.794111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b78805af-8b4f-4ea8-adec-78e9e0d2c20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:03.921036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 679/1365] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:03.961049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd2fc0-191a-472c-8c68-012c218625de None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:03.961592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd2fc0-191a-472c-8c68-012c218625de None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:03.962107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd2fc0-191a-472c-8c68-012c218625de None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:03.963663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 687/1366] 10.222.0.50 () {54 vars in 962 bytes} [Mon Aug 8 18:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:03.969838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d7eaf-1c6d-4a00-a5b4-77a349dcbc90 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:03.970199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d7eaf-1c6d-4a00-a5b4-77a349dcbc90 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:03.970541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d7eaf-1c6d-4a00-a5b4-77a349dcbc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:04.054771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8d3d7eaf-1c6d-4a00-a5b4-77a349dcbc90 None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:04.055981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8d3d7eaf-1c6d-4a00-a5b4-77a349dcbc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:04.092464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:04.189694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 680/1367] 10.222.0.50 () {58 vars in 1059 bytes} [Mon Aug 8 18:02:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:04.197594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd2fc0-191a-472c-8c68-012c218625de None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:04.199397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-50fd2fc0-191a-472c-8c68-012c218625de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:04.359207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd2fc0-191a-472c-8c68-012c218625de None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:04.381285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:04.478154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1029-6a13-4ad0-b96f-182c0970a30b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:04.478660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1029-6a13-4ad0-b96f-182c0970a30b service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:04.478940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1029-6a13-4ad0-b96f-182c0970a30b service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:04.479197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1029-6a13-4ad0-b96f-182c0970a30b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:04.482260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7c9e1029-6a13-4ad0-b96f-182c0970a30b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:04.578310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e1029-6a13-4ad0-b96f-182c0970a30b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:04.579930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e1029-6a13-4ad0-b96f-182c0970a30b service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:04.600034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 688/1368] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:05.569182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc00d64-6f53-46cd-9a94-c5b1cd0a84b7 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:05.569734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc00d64-6f53-46cd-9a94-c5b1cd0a84b7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:05.569734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc00d64-6f53-46cd-9a94-c5b1cd0a84b7 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:05.571358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 681/1369] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:05.577105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5019d6-99dd-4a44-a49d-fd06fbe3a51e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:05.577576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5019d6-99dd-4a44-a49d-fd06fbe3a51e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:05.577657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5019d6-99dd-4a44-a49d-fd06fbe3a51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:05.611172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:05.647692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2d5019d6-99dd-4a44-a49d-fd06fbe3a51e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:05.648941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2d5019d6-99dd-4a44-a49d-fd06fbe3a51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:05.756606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 689/1370] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:06.098255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf289-8346-4587-aab8-e4b8e1a7a449 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:06.098255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf289-8346-4587-aab8-e4b8e1a7a449 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:06.098255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf289-8346-4587-aab8-e4b8e1a7a449 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:06.100563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 682/1371] 10.222.0.50 () {54 vars in 964 bytes} [Mon Aug 8 18:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:06.104966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ecd2b-410d-46d8-9f1f-e12ab4a27883 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:06.104966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ecd2b-410d-46d8-9f1f-e12ab4a27883 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:06.104966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ecd2b-410d-46d8-9f1f-e12ab4a27883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:06.182952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-024ecd2b-410d-46d8-9f1f-e12ab4a27883 None None] MFA Rules not processed for user `5b4aef937fb047f6869a156a68e7bf2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:06.184417 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-024ecd2b-410d-46d8-9f1f-e12ab4a27883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:06.235682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:06.265851 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 690/1372] 10.222.0.50 () {58 vars in 1061 bytes} [Mon Aug 8 18:02:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:02:06.274692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf289-8346-4587-aab8-e4b8e1a7a449 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:06.276381 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-364bf289-8346-4587-aab8-e4b8e1a7a449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:06.391677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf289-8346-4587-aab8-e4b8e1a7a449 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:06.436058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:06.474955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bf0e9-cfda-4dc3-8199-880af691a3d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:06.475558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bf0e9-cfda-4dc3-8199-880af691a3d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:06.475830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bf0e9-cfda-4dc3-8199-880af691a3d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:06.476101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bf0e9-cfda-4dc3-8199-880af691a3d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:06.477746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c23bf0e9-cfda-4dc3-8199-880af691a3d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:06.601292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bf0e9-cfda-4dc3-8199-880af691a3d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:06.602864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bf0e9-cfda-4dc3-8199-880af691a3d4 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:06.622530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 683/1373] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:06.841465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed53c9-2239-45ae-9cf7-98efbc230cb4 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:06.841953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed53c9-2239-45ae-9cf7-98efbc230cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:06.841953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed53c9-2239-45ae-9cf7-98efbc230cb4 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:06.847167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 691/1374] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:06.848466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31997235-de03-4767-81de-8a92e8a46642 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:06.848700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31997235-de03-4767-81de-8a92e8a46642 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:06.849309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31997235-de03-4767-81de-8a92e8a46642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:06.923456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-31997235-de03-4767-81de-8a92e8a46642 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:06.924868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-31997235-de03-4767-81de-8a92e8a46642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:06.947278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:07.028498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 684/1375] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:07.072135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed53c9-2239-45ae-9cf7-98efbc230cb4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:07.074369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4aed53c9-2239-45ae-9cf7-98efbc230cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:07.219410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed53c9-2239-45ae-9cf7-98efbc230cb4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:07.273386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:07.334651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830f34d-c4ca-452f-a3d1-a6f696720b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:07.335269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830f34d-c4ca-452f-a3d1-a6f696720b3a service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:07.335601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830f34d-c4ca-452f-a3d1-a6f696720b3a service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:07.335866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830f34d-c4ca-452f-a3d1-a6f696720b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:07.337481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6830f34d-c4ca-452f-a3d1-a6f696720b3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:07.428380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830f34d-c4ca-452f-a3d1-a6f696720b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:07.430009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830f34d-c4ca-452f-a3d1-a6f696720b3a service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:07.449433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 692/1376] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:09.590329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ad48-1bcd-42c2-b514-b8ee862f0350 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:09.590762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ad48-1bcd-42c2-b514-b8ee862f0350 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:09.590927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ad48-1bcd-42c2-b514-b8ee862f0350 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:09.592433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 685/1377] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:09.598002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6176045-3881-4a8b-9a78-ffe4cd0e53ec None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:09.598407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6176045-3881-4a8b-9a78-ffe4cd0e53ec None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:09.598547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6176045-3881-4a8b-9a78-ffe4cd0e53ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:09.671689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d6176045-3881-4a8b-9a78-ffe4cd0e53ec None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:09.672910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d6176045-3881-4a8b-9a78-ffe4cd0e53ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:09.679862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:09.781142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 693/1378] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:09.826550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17ad48-1bcd-42c2-b514-b8ee862f0350 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:09.828261 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ea17ad48-1bcd-42c2-b514-b8ee862f0350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:09.971597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17ad48-1bcd-42c2-b514-b8ee862f0350 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:09.975530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:10.095714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92211724-d7ab-477d-8248-3eba4745e328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:10.096298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92211724-d7ab-477d-8248-3eba4745e328 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:10.096589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92211724-d7ab-477d-8248-3eba4745e328 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:10.096852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92211724-d7ab-477d-8248-3eba4745e328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:10.098455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-92211724-d7ab-477d-8248-3eba4745e328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:10.213147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92211724-d7ab-477d-8248-3eba4745e328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:10.215462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92211724-d7ab-477d-8248-3eba4745e328 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:10.237025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 686/1379] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:12.450096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffd767-a6bb-4eff-bb6b-d3b7ce40b959 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:12.450651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffd767-a6bb-4eff-bb6b-d3b7ce40b959 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:12.450651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffd767-a6bb-4eff-bb6b-d3b7ce40b959 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:12.452305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 694/1380] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:12.457584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc462b-c143-4d7d-8dfc-293e04b1db0e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:12.458013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc462b-c143-4d7d-8dfc-293e04b1db0e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:12.458097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc462b-c143-4d7d-8dfc-293e04b1db0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:12.469559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:12.526663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-20dc462b-c143-4d7d-8dfc-293e04b1db0e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:12.527837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-20dc462b-c143-4d7d-8dfc-293e04b1db0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:12.637245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 687/1381] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:12.681222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffd767-a6bb-4eff-bb6b-d3b7ce40b959 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:12.683347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2bffd767-a6bb-4eff-bb6b-d3b7ce40b959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:12.837052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:12.848085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffd767-a6bb-4eff-bb6b-d3b7ce40b959 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:12.965570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7bb8bf-273c-43c9-be23-ed7c6126804a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:12.966152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bb8bf-273c-43c9-be23-ed7c6126804a service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:12.966424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bb8bf-273c-43c9-be23-ed7c6126804a service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:12.966690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bb8bf-273c-43c9-be23-ed7c6126804a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:12.968273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3b7bb8bf-273c-43c9-be23-ed7c6126804a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:13.072232 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bb8bf-273c-43c9-be23-ed7c6126804a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:13.074390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bb8bf-273c-43c9-be23-ed7c6126804a service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:13.087924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:13.098447 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 695/1382] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:15.293690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf86980-ca36-459d-95ca-32728e29697d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:15.294475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf86980-ca36-459d-95ca-32728e29697d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:15.294475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf86980-ca36-459d-95ca-32728e29697d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:15.295737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 688/1383] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:15.302068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63124a-5538-4fa4-8cfa-cfd0772545b8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:15.302536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63124a-5538-4fa4-8cfa-cfd0772545b8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:15.302640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63124a-5538-4fa4-8cfa-cfd0772545b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:15.369261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3c63124a-5538-4fa4-8cfa-cfd0772545b8 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:15.371092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3c63124a-5538-4fa4-8cfa-cfd0772545b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:15.482046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 696/1384] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:15.526430 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf86980-ca36-459d-95ca-32728e29697d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:15.527912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9bf86980-ca36-459d-95ca-32728e29697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:15.633973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:15.701829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf86980-ca36-459d-95ca-32728e29697d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:15.835285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d54a48-9da5-487a-a9a6-1c2179cafac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:15.835801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d54a48-9da5-487a-a9a6-1c2179cafac9 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:15.836071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d54a48-9da5-487a-a9a6-1c2179cafac9 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:15.836334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d54a48-9da5-487a-a9a6-1c2179cafac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:15.837940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-66d54a48-9da5-487a-a9a6-1c2179cafac9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:15.888435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:15.935110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d54a48-9da5-487a-a9a6-1c2179cafac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:15.936714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d54a48-9da5-487a-a9a6-1c2179cafac9 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:15.954543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 689/1385] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:18.433818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffabb06-5083-4c56-8702-4df2b156607c None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:18.434418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffabb06-5083-4c56-8702-4df2b156607c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:18.434418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffabb06-5083-4c56-8702-4df2b156607c None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:18.435760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 697/1386] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:18.441357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f4b1a-ef05-49a8-acbd-3178b223e69e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:18.441799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f4b1a-ef05-49a8-acbd-3178b223e69e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:18.442139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f4b1a-ef05-49a8-acbd-3178b223e69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:18.519298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-991f4b1a-ef05-49a8-acbd-3178b223e69e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:18.520427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-991f4b1a-ef05-49a8-acbd-3178b223e69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:18.620807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 690/1387] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:18.665449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffabb06-5083-4c56-8702-4df2b156607c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:18.668036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3ffabb06-5083-4c56-8702-4df2b156607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:18.786706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:18.887650 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffabb06-5083-4c56-8702-4df2b156607c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:19.011604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdce59d-63b0-41d2-add1-f62c2f48cda3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:19.012207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdce59d-63b0-41d2-add1-f62c2f48cda3 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:19.012478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdce59d-63b0-41d2-add1-f62c2f48cda3 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:19.012846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdce59d-63b0-41d2-add1-f62c2f48cda3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:19.014462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1bdce59d-63b0-41d2-add1-f62c2f48cda3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:19.052721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:19.105480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdce59d-63b0-41d2-add1-f62c2f48cda3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:19.107112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdce59d-63b0-41d2-add1-f62c2f48cda3 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:19.127110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 698/1388] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:21.294637 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d694a-4b0b-4506-8200-9050848eda26 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:21.295143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d694a-4b0b-4506-8200-9050848eda26 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:21.295143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d694a-4b0b-4506-8200-9050848eda26 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:21.296599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 691/1389] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:21.301476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a047373-d11b-44bd-b238-aaec89dee80c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:21.301870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a047373-d11b-44bd-b238-aaec89dee80c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:21.302142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a047373-d11b-44bd-b238-aaec89dee80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:21.368107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4a047373-d11b-44bd-b238-aaec89dee80c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:21.369319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4a047373-d11b-44bd-b238-aaec89dee80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:21.469944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 699/1390] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:21.514727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d694a-4b0b-4506-8200-9050848eda26 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:21.516281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-357d694a-4b0b-4506-8200-9050848eda26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:21.546506 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:21.663229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d694a-4b0b-4506-8200-9050848eda26 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:21.768006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:21.779268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf4f58-796d-4a8a-b57d-08e31646db63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:21.779913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf4f58-796d-4a8a-b57d-08e31646db63 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:21.780183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf4f58-796d-4a8a-b57d-08e31646db63 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:21.780443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf4f58-796d-4a8a-b57d-08e31646db63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:21.782058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f3bf4f58-796d-4a8a-b57d-08e31646db63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:21.874488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf4f58-796d-4a8a-b57d-08e31646db63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:21.876089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf4f58-796d-4a8a-b57d-08e31646db63 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:21.894231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 692/1391] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:02:22.011879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eec769-3798-4643-914f-5ac9f949a186 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:22.012157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eec769-3798-4643-914f-5ac9f949a186 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:22.012423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eec769-3798-4643-914f-5ac9f949a186 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:22.013836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 700/1392] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:22.019073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dac04-b991-4560-b435-7b08ebf847cd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:22.019348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dac04-b991-4560-b435-7b08ebf847cd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:22.019631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dac04-b991-4560-b435-7b08ebf847cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:22.088685 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2d6dac04-b991-4560-b435-7b08ebf847cd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:22.089916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2d6dac04-b991-4560-b435-7b08ebf847cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:22.155661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:22.198923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 693/1393] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:22.256188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b84ea-70d2-491e-9acf-8c6ea7349e7b None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:22.256464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b84ea-70d2-491e-9acf-8c6ea7349e7b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:22.256761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b84ea-70d2-491e-9acf-8c6ea7349e7b None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:22.258093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 701/1394] 10.222.0.50 () {54 vars in 967 bytes} [Mon Aug 8 18:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:22.265400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089d8d2-82fe-45a6-9292-a02cffd2f305 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:22.265685 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089d8d2-82fe-45a6-9292-a02cffd2f305 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:22.265957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089d8d2-82fe-45a6-9292-a02cffd2f305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:22.341198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3089d8d2-82fe-45a6-9292-a02cffd2f305 None None] MFA Rules not processed for user `0e95d295337b4472be946d252ac3bd7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:22.342333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3089d8d2-82fe-45a6-9292-a02cffd2f305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:22.417890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 694/1395] 10.222.0.50 () {58 vars in 1064 bytes} [Mon Aug 8 18:02:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:22.425377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b84ea-70d2-491e-9acf-8c6ea7349e7b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:22.426971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3f8b84ea-70d2-491e-9acf-8c6ea7349e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:22.440545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:22.535991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b84ea-70d2-491e-9acf-8c6ea7349e7b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:22.613323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0ab73-3afc-43e8-a9fe-f00f25c4b2a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e95d295337b4472be946d252ac3bd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:22.613861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0ab73-3afc-43e8-a9fe-f00f25c4b2a4 service glance] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:22.614135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0ab73-3afc-43e8-a9fe-f00f25c4b2a4 service glance] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:22.614402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0ab73-3afc-43e8-a9fe-f00f25c4b2a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:22.616001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-85d0ab73-3afc-43e8-a9fe-f00f25c4b2a4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:22.683620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:22.726738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0ab73-3afc-43e8-a9fe-f00f25c4b2a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:22.728185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0ab73-3afc-43e8-a9fe-f00f25c4b2a4 service glance] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:22.746227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 702/1396] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:23.647224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6a6f-e1b3-4fff-974a-1d99f2fe9344 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:23.647694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6a6f-e1b3-4fff-974a-1d99f2fe9344 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:23.647809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6a6f-e1b3-4fff-974a-1d99f2fe9344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:23.725813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-952a6a6f-e1b3-4fff-974a-1d99f2fe9344 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:23.727024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-952a6a6f-e1b3-4fff-974a-1d99f2fe9344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:23.736069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:23.815972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 695/1397] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:02:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:23.863895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030310e1-ee45-45b5-9833-05349882df4f None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:23.864369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030310e1-ee45-45b5-9833-05349882df4f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:23.864474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030310e1-ee45-45b5-9833-05349882df4f None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:23.865872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 703/1398] 10.222.0.50 () {54 vars in 943 bytes} [Mon Aug 8 18:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:23.872146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cf967-3914-4722-bd66-529c612089c1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:23.872434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cf967-3914-4722-bd66-529c612089c1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:23.874166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cf967-3914-4722-bd66-529c612089c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:23.952042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0e1cf967-3914-4722-bd66-529c612089c1 None None] MFA Rules not processed for user `3375c3134ebe473cb5da29f083321cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:23.953542 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0e1cf967-3914-4722-bd66-529c612089c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:24.030877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1a6f3-99ef-4c35-8210-41d57d6556f0 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:24.031228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1a6f3-99ef-4c35-8210-41d57d6556f0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:24.031424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1a6f3-99ef-4c35-8210-41d57d6556f0 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:24.032787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 704/1399] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 18:02:24.037479 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e06486-4da1-45d8-9b19-46b11df3b213 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:24.037748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:24.037873 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e06486-4da1-45d8-9b19-46b11df3b213 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:24.038015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e06486-4da1-45d8-9b19-46b11df3b213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:24.061511 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 696/1400] 10.222.0.50 () {58 vars in 1040 bytes} [Mon Aug 8 18:02:23 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:02:24.071001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cf967-3914-4722-bd66-529c612089c1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:24.081534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cf967-3914-4722-bd66-529c612089c1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:24.105421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-83e06486-4da1-45d8-9b19-46b11df3b213 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:24.106540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-83e06486-4da1-45d8-9b19-46b11df3b213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:24.190560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55814408-4b42-43e5-97bc-ac999d247c4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:24.191161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55814408-4b42-43e5-97bc-ac999d247c4b service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:24.191433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55814408-4b42-43e5-97bc-ac999d247c4b service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:24.191702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55814408-4b42-43e5-97bc-ac999d247c4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:24.193407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-55814408-4b42-43e5-97bc-ac999d247c4b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:24.207102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 705/1401] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:02:24.250839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e06486-4da1-45d8-9b19-46b11df3b213 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:24.252041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-83e06486-4da1-45d8-9b19-46b11df3b213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:24.277400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55814408-4b42-43e5-97bc-ac999d247c4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:24.278997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55814408-4b42-43e5-97bc-ac999d247c4b service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:24.281852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:24.287078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 697/1402] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:02:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 18:02:24.368972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e06486-4da1-45d8-9b19-46b11df3b213 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:24.480199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c482e81-ba72-40de-9b03-213d884d25c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:24.480854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c482e81-ba72-40de-9b03-213d884d25c2 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:24.481122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c482e81-ba72-40de-9b03-213d884d25c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:24.481381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c482e81-ba72-40de-9b03-213d884d25c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:24.482944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2c482e81-ba72-40de-9b03-213d884d25c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:24.501315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:24.572727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c482e81-ba72-40de-9b03-213d884d25c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:24.574929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c482e81-ba72-40de-9b03-213d884d25c2 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:24.594136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 706/1403] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:26.814806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fa306-e0f4-47c9-afbb-fc2661cda34c None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:26.815273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fa306-e0f4-47c9-afbb-fc2661cda34c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:26.815493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fa306-e0f4-47c9-afbb-fc2661cda34c None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:26.820561 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 698/1404] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:26.822640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a528ff-8466-4c51-9dc4-9bec64c20e7c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:26.823583 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a528ff-8466-4c51-9dc4-9bec64c20e7c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:26.823784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a528ff-8466-4c51-9dc4-9bec64c20e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:26.900449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-22a528ff-8466-4c51-9dc4-9bec64c20e7c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:26.901734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-22a528ff-8466-4c51-9dc4-9bec64c20e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:27.007632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:27.015122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 707/1405] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:27.061621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fa306-e0f4-47c9-afbb-fc2661cda34c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:27.063403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bc7fa306-e0f4-47c9-afbb-fc2661cda34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:27.074084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:27.227182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fa306-e0f4-47c9-afbb-fc2661cda34c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:27.320262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:27.355904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552813d6-6533-4096-9579-efda6b87fc57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:27.356643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552813d6-6533-4096-9579-efda6b87fc57 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:27.356981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552813d6-6533-4096-9579-efda6b87fc57 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:27.357317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552813d6-6533-4096-9579-efda6b87fc57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:27.358969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-552813d6-6533-4096-9579-efda6b87fc57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:27.458315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552813d6-6533-4096-9579-efda6b87fc57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:27.459913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552813d6-6533-4096-9579-efda6b87fc57 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:27.481521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 699/1406] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:02:29.773817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9156bba-9404-40e7-aac8-40f01b90a0cd None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:29.774449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9156bba-9404-40e7-aac8-40f01b90a0cd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:29.774623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9156bba-9404-40e7-aac8-40f01b90a0cd None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:29.776987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 708/1407] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:29.781769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50978b42-ab63-4328-ad88-ba9dce73706f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:29.782226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50978b42-ab63-4328-ad88-ba9dce73706f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:29.782328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50978b42-ab63-4328-ad88-ba9dce73706f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:29.861538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-50978b42-ab63-4328-ad88-ba9dce73706f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:29.863675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-50978b42-ab63-4328-ad88-ba9dce73706f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:29.914610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:29.972175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 700/1408] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:02:30.016555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9156bba-9404-40e7-aac8-40f01b90a0cd None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:30.018122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e9156bba-9404-40e7-aac8-40f01b90a0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:30.186526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9156bba-9404-40e7-aac8-40f01b90a0cd None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:30.260706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:30.314136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085616d2-de1e-4b2f-a839-51b89acea422 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:30.314739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085616d2-de1e-4b2f-a839-51b89acea422 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:30.315004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085616d2-de1e-4b2f-a839-51b89acea422 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:30.315268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085616d2-de1e-4b2f-a839-51b89acea422 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:30.316942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-085616d2-de1e-4b2f-a839-51b89acea422 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:30.417307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085616d2-de1e-4b2f-a839-51b89acea422 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:30.419400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085616d2-de1e-4b2f-a839-51b89acea422 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:30.440296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 709/1409] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:39.969970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=23 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:40.437134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=64 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:47.596991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbb9be-0297-48d8-8556-193ad146be39 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:47.597403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbb9be-0297-48d8-8556-193ad146be39 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:47.597530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbb9be-0297-48d8-8556-193ad146be39 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:47.599780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 701/1410] 10.222.0.50 () {54 vars in 918 bytes} [Mon Aug 8 18:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:47.604110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e3f9a-44f7-4d46-b768-b91bb6cc36c8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:47.604552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e3f9a-44f7-4d46-b768-b91bb6cc36c8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:47.604670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e3f9a-44f7-4d46-b768-b91bb6cc36c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:47.680241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8f9e3f9a-44f7-4d46-b768-b91bb6cc36c8 None None] MFA Rules not processed for user `b81bdfeedcba4b0c93f13d14318b2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:47.682891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8f9e3f9a-44f7-4d46-b768-b91bb6cc36c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:47.787348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 710/1411] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Aug 8 18:02:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:47.801788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856536f3-cbb6-49dd-a66a-4df93b657151 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:47.802485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856536f3-cbb6-49dd-a66a-4df93b657151 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:47.802759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856536f3-cbb6-49dd-a66a-4df93b657151 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:47.804087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 702/1412] 10.222.0.50 () {54 vars in 918 bytes} [Mon Aug 8 18:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:47.809242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af8604-26eb-4d59-aa9f-26821428ce76 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:47.809529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af8604-26eb-4d59-aa9f-26821428ce76 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:47.809805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af8604-26eb-4d59-aa9f-26821428ce76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:47.881571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:47.885022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d7af8604-26eb-4d59-aa9f-26821428ce76 None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:47.886229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d7af8604-26eb-4d59-aa9f-26821428ce76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:48.003744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 711/1413] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Aug 8 18:02:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:50.451387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6792c-1d17-4729-b2a3-c9e553a73386 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:50.451908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6792c-1d17-4729-b2a3-c9e553a73386 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:50.451908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6792c-1d17-4729-b2a3-c9e553a73386 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:50.453450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:50.458567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2559a8-983f-4aaa-91a0-6a4345652820 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:50.458946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2559a8-983f-4aaa-91a0-6a4345652820 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:50.459112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2559a8-983f-4aaa-91a0-6a4345652820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:50.525898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6c2559a8-983f-4aaa-91a0-6a4345652820 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:50.527090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6c2559a8-983f-4aaa-91a0-6a4345652820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:50.580813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:50.622188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:50.665000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6792c-1d17-4729-b2a3-c9e553a73386 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:50.666507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ffb6792c-1d17-4729-b2a3-c9e553a73386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:50.812640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6792c-1d17-4729-b2a3-c9e553a73386 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:50.883481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:50.923206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf963cbf-bcd8-4546-a8e0-352f05ec4051 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:50.923806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf963cbf-bcd8-4546-a8e0-352f05ec4051 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:50.924077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf963cbf-bcd8-4546-a8e0-352f05ec4051 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:50.924343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf963cbf-bcd8-4546-a8e0-352f05ec4051 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:50.925968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bf963cbf-bcd8-4546-a8e0-352f05ec4051 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:51.014270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf963cbf-bcd8-4546-a8e0-352f05ec4051 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:51.015859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf963cbf-bcd8-4546-a8e0-352f05ec4051 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:51.033372 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 704/1416] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:51.928170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de1a75-b1d2-4753-bc6c-12869114677d None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:51.928733 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de1a75-b1d2-4753-bc6c-12869114677d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:51.928733 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de1a75-b1d2-4753-bc6c-12869114677d None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:51.929991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:51.935448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b24e5-480a-4474-8bf3-4a7adf4d26f3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:51.935882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b24e5-480a-4474-8bf3-4a7adf4d26f3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:51.936003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b24e5-480a-4474-8bf3-4a7adf4d26f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:52.010459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-175b24e5-480a-4474-8bf3-4a7adf4d26f3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:52.011683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-175b24e5-480a-4474-8bf3-4a7adf4d26f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:52.037160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:52.104403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:52.251375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de1a75-b1d2-4753-bc6c-12869114677d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:52.252831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-55de1a75-b1d2-4753-bc6c-12869114677d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:52.392020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de1a75-b1d2-4753-bc6c-12869114677d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:52.430940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:52.501568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f65dd-5eb4-4644-9fcf-a584235dbfea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:52.502085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f65dd-5eb4-4644-9fcf-a584235dbfea service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:52.502382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f65dd-5eb4-4644-9fcf-a584235dbfea service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:52.502638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f65dd-5eb4-4644-9fcf-a584235dbfea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:52.504196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c94f65dd-5eb4-4644-9fcf-a584235dbfea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:52.591125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f65dd-5eb4-4644-9fcf-a584235dbfea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:52.592735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f65dd-5eb4-4644-9fcf-a584235dbfea service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:52.610416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 714/1419] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:53.485493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cca1f6-a54d-4fb1-af10-72db6cd256eb None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:53.486061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cca1f6-a54d-4fb1-af10-72db6cd256eb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:53.486061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cca1f6-a54d-4fb1-af10-72db6cd256eb None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:53.487353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:53.492189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb82c46-a929-405b-8fc1-3cb57e941c03 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:53.492585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb82c46-a929-405b-8fc1-3cb57e941c03 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:53.492910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb82c46-a929-405b-8fc1-3cb57e941c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:53.548658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:53.569752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6bb82c46-a929-405b-8fc1-3cb57e941c03 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:53.570950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6bb82c46-a929-405b-8fc1-3cb57e941c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:53.670692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:53.826183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cca1f6-a54d-4fb1-af10-72db6cd256eb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:53.827956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d1cca1f6-a54d-4fb1-af10-72db6cd256eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:53.970506 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cca1f6-a54d-4fb1-af10-72db6cd256eb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:53.982322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:54.081917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f8d87-1047-4180-abd9-02e4641d38b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:54.082441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f8d87-1047-4180-abd9-02e4641d38b2 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:54.082707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f8d87-1047-4180-abd9-02e4641d38b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:54.082966 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f8d87-1047-4180-abd9-02e4641d38b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:54.084573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7b6f8d87-1047-4180-abd9-02e4641d38b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:54.172746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f8d87-1047-4180-abd9-02e4641d38b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:54.174342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f8d87-1047-4180-abd9-02e4641d38b2 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:54.192301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 707/1422] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:55.055276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b0376-59e8-499f-b6d3-a29031f9d455 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:55.055874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b0376-59e8-499f-b6d3-a29031f9d455 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:55.055874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b0376-59e8-499f-b6d3-a29031f9d455 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:55.057098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:55.061868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63de40-478d-4359-a223-0d6381b154fc None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:55.062307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63de40-478d-4359-a223-0d6381b154fc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:55.062402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63de40-478d-4359-a223-0d6381b154fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:55.082289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:55.137103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0e63de40-478d-4359-a223-0d6381b154fc None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:55.138335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0e63de40-478d-4359-a223-0d6381b154fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:55.232105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:55.384008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b0376-59e8-499f-b6d3-a29031f9d455 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:55.386049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fd0b0376-59e8-499f-b6d3-a29031f9d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:55.494114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:55.530235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b0376-59e8-499f-b6d3-a29031f9d455 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:55.643411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400a629-2979-4e9c-87bd-503b0ab30f74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:55.643946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400a629-2979-4e9c-87bd-503b0ab30f74 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:55.644219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400a629-2979-4e9c-87bd-503b0ab30f74 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:55.644484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400a629-2979-4e9c-87bd-503b0ab30f74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:55.646098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6400a629-2979-4e9c-87bd-503b0ab30f74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:55.720131 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:55.733668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400a629-2979-4e9c-87bd-503b0ab30f74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:55.735433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400a629-2979-4e9c-87bd-503b0ab30f74 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:55.755046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 717/1425] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:56.624971 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a6e0b-2c7a-40e1-a804-f676a82ac3bc None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:56.625514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a6e0b-2c7a-40e1-a804-f676a82ac3bc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:56.625514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a6e0b-2c7a-40e1-a804-f676a82ac3bc None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:56.626838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:56.631767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7884620-0d99-481c-b0b6-5ec4cd75e42e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:56.632192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7884620-0d99-481c-b0b6-5ec4cd75e42e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:56.632306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7884620-0d99-481c-b0b6-5ec4cd75e42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:56.712750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e7884620-0d99-481c-b0b6-5ec4cd75e42e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:56.714012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e7884620-0d99-481c-b0b6-5ec4cd75e42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:56.819222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:56.974172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a6e0b-2c7a-40e1-a804-f676a82ac3bc None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:56.975603 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d11a6e0b-2c7a-40e1-a804-f676a82ac3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:57.059171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:57.121426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a6e0b-2c7a-40e1-a804-f676a82ac3bc None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:57.230695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757ef20-6309-461d-868f-54f56b35fdcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:57.231210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757ef20-6309-461d-868f-54f56b35fdcc service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:57.231477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757ef20-6309-461d-868f-54f56b35fdcc service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:57.231737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757ef20-6309-461d-868f-54f56b35fdcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:57.233334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e757ef20-6309-461d-868f-54f56b35fdcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:57.279621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:57.320299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757ef20-6309-461d-868f-54f56b35fdcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:57.321973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757ef20-6309-461d-868f-54f56b35fdcc service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:57.339437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 710/1428] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:58.192644 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99252-4299-44e8-9986-c2c0c8b6d65e None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:58.193128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99252-4299-44e8-9986-c2c0c8b6d65e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:58.193241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99252-4299-44e8-9986-c2c0c8b6d65e None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:58.194533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:58.200388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d420-920a-409e-9baa-296afea6bef5 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:58.200877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d420-920a-409e-9baa-296afea6bef5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:58.200986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d420-920a-409e-9baa-296afea6bef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:58.276278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1896d420-920a-409e-9baa-296afea6bef5 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:58.277561 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1896d420-920a-409e-9baa-296afea6bef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:58.371037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:02:58.519408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f99252-4299-44e8-9986-c2c0c8b6d65e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:02:58.520960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d8f99252-4299-44e8-9986-c2c0c8b6d65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:58.568703 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:58.663084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f99252-4299-44e8-9986-c2c0c8b6d65e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:02:58.780003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88908208-2ef7-445f-8953-f88a80731d56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:02:58.780530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88908208-2ef7-445f-8953-f88a80731d56 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:58.780812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88908208-2ef7-445f-8953-f88a80731d56 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:58.781074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88908208-2ef7-445f-8953-f88a80731d56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:58.782652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-88908208-2ef7-445f-8953-f88a80731d56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:58.789528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:58.872239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88908208-2ef7-445f-8953-f88a80731d56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:02:58.873887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88908208-2ef7-445f-8953-f88a80731d56 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:02:58.891600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 720/1431] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:02:59.786912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc428b5-14ba-4303-8515-38a84bbbd11e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:59.787410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc428b5-14ba-4303-8515-38a84bbbd11e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:59.787538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc428b5-14ba-4303-8515-38a84bbbd11e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:59.788855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:02:59.793819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa285f-c81a-4fea-bf77-61ccb89fb16d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:02:59.794261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa285f-c81a-4fea-bf77-61ccb89fb16d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:02:59.794362 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa285f-c81a-4fea-bf77-61ccb89fb16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:02:59.872268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-77fa285f-c81a-4fea-bf77-61ccb89fb16d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:02:59.873423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-77fa285f-c81a-4fea-bf77-61ccb89fb16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:02:59.939995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:02:59.968466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:00.119163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc428b5-14ba-4303-8515-38a84bbbd11e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:00.120714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0dc428b5-14ba-4303-8515-38a84bbbd11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:00.137604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:00.260388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc428b5-14ba-4303-8515-38a84bbbd11e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:00.347583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:00.375147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4dd4-0e41-490f-9371-7d6362cb856d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:00.375750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4dd4-0e41-490f-9371-7d6362cb856d service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:00.376024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4dd4-0e41-490f-9371-7d6362cb856d service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:00.376416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4dd4-0e41-490f-9371-7d6362cb856d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:00.378081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-405e4dd4-0e41-490f-9371-7d6362cb856d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:00.467261 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e4dd4-0e41-490f-9371-7d6362cb856d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:00.468884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e4dd4-0e41-490f-9371-7d6362cb856d service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:00.487043 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 713/1434] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:01.596263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9658019-64a8-4a30-b3d5-bfe27c204195 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:01.596898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9658019-64a8-4a30-b3d5-bfe27c204195 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:01.596898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9658019-64a8-4a30-b3d5-bfe27c204195 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:01.598157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:01.603396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503c5e3-dbb0-4261-a194-9eab37a351d3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:01.603841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503c5e3-dbb0-4261-a194-9eab37a351d3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:01.603946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503c5e3-dbb0-4261-a194-9eab37a351d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:01.677723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a503c5e3-dbb0-4261-a194-9eab37a351d3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:01.678925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a503c5e3-dbb0-4261-a194-9eab37a351d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:01.722700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:01.784582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:01.934184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9658019-64a8-4a30-b3d5-bfe27c204195 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:01.935576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f9658019-64a8-4a30-b3d5-bfe27c204195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:02.079822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9658019-64a8-4a30-b3d5-bfe27c204195 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:02.133066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:02.191033 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f2bdd-6604-4f21-b967-a882cf573922 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:02.191561 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f2bdd-6604-4f21-b967-a882cf573922 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:02.191837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f2bdd-6604-4f21-b967-a882cf573922 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:02.192101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f2bdd-6604-4f21-b967-a882cf573922 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:02.193709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-607f2bdd-6604-4f21-b967-a882cf573922 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:02.284262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f2bdd-6604-4f21-b967-a882cf573922 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:02.285931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f2bdd-6604-4f21-b967-a882cf573922 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:02.303923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 723/1437] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:03.188200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195a81e-d6fa-40ab-b80d-94083483daf4 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:03.188775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195a81e-d6fa-40ab-b80d-94083483daf4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:03.188775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195a81e-d6fa-40ab-b80d-94083483daf4 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:03.190019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:03.194964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a76780-5682-43f9-968c-3183b7458987 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:03.195378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a76780-5682-43f9-968c-3183b7458987 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:03.195491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a76780-5682-43f9-968c-3183b7458987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:03.271783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-88a76780-5682-43f9-968c-3183b7458987 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:03.273000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-88a76780-5682-43f9-968c-3183b7458987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:03.274948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:03.367408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:03.526235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195a81e-d6fa-40ab-b80d-94083483daf4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:03.527739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b195a81e-d6fa-40ab-b80d-94083483daf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:03.670962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195a81e-d6fa-40ab-b80d-94083483daf4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:03.697796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:03.787764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec3dcd-9151-4762-94de-6cabce0bf5ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:03.788424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec3dcd-9151-4762-94de-6cabce0bf5ab service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:03.788726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec3dcd-9151-4762-94de-6cabce0bf5ab service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:03.788989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec3dcd-9151-4762-94de-6cabce0bf5ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:03.791096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5dec3dcd-9151-4762-94de-6cabce0bf5ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:03.883386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec3dcd-9151-4762-94de-6cabce0bf5ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:03.885041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec3dcd-9151-4762-94de-6cabce0bf5ab service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:03.904797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 716/1440] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:05.814174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ad795-745a-4152-876f-4334c980978a None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:05.814615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ad795-745a-4152-876f-4334c980978a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:05.814724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ad795-745a-4152-876f-4334c980978a None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:05.816050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:05.821179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69038900-73ba-4436-9c8b-073dd4be9db3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:05.821610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69038900-73ba-4436-9c8b-073dd4be9db3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:05.821724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69038900-73ba-4436-9c8b-073dd4be9db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:05.863912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:05.889459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-69038900-73ba-4436-9c8b-073dd4be9db3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:05.890645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-69038900-73ba-4436-9c8b-073dd4be9db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:05.985580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:06.024754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ad795-745a-4152-876f-4334c980978a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:06.026308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-020ad795-745a-4152-876f-4334c980978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:06.128828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ad795-745a-4152-876f-4334c980978a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:06.160032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:06.201606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6825e0-c790-49be-8972-54450c0fdbce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:06.202200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6825e0-c790-49be-8972-54450c0fdbce service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:06.202730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6825e0-c790-49be-8972-54450c0fdbce service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:06.203001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6825e0-c790-49be-8972-54450c0fdbce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:06.204613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7d6825e0-c790-49be-8972-54450c0fdbce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:06.310691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6825e0-c790-49be-8972-54450c0fdbce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:06.312115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6825e0-c790-49be-8972-54450c0fdbce service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:06.330653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 726/1443] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:08.460724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1b6fc-6d04-4917-918a-1b4a83d12e58 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:08.461294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1b6fc-6d04-4917-918a-1b4a83d12e58 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:08.461294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1b6fc-6d04-4917-918a-1b4a83d12e58 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:08.463231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:08.468308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a796635-dfed-4027-b02a-7ef771792e85 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:08.468786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a796635-dfed-4027-b02a-7ef771792e85 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:08.468920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a796635-dfed-4027-b02a-7ef771792e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:08.655120 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2a796635-dfed-4027-b02a-7ef771792e85 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:08.656325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2a796635-dfed-4027-b02a-7ef771792e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:08.668771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:08.751577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:08.789804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1b6fc-6d04-4917-918a-1b4a83d12e58 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:08.791317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-86c1b6fc-6d04-4917-918a-1b4a83d12e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:08.892735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1b6fc-6d04-4917-918a-1b4a83d12e58 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:08.904446 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:08.963699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb363b3-2007-4660-b818-ff2285382847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:08.964286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb363b3-2007-4660-b818-ff2285382847 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:08.964580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb363b3-2007-4660-b818-ff2285382847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:08.964843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb363b3-2007-4660-b818-ff2285382847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:08.966421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5fb363b3-2007-4660-b818-ff2285382847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:09.074272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb363b3-2007-4660-b818-ff2285382847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:09.075712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb363b3-2007-4660-b818-ff2285382847 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:09.095125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 719/1446] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:11.445402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998ef1a-4f02-4420-8177-cb955f064b43 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:11.446026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998ef1a-4f02-4420-8177-cb955f064b43 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:11.446026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998ef1a-4f02-4420-8177-cb955f064b43 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:11.447771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:11.453261 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef871b7-634e-414c-bad0-eb80f0cc7013 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:11.453703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef871b7-634e-414c-bad0-eb80f0cc7013 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:11.453804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef871b7-634e-414c-bad0-eb80f0cc7013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:11.619459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:11.638417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1ef871b7-634e-414c-bad0-eb80f0cc7013 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:11.639821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1ef871b7-634e-414c-bad0-eb80f0cc7013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:11.737259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:11.777334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998ef1a-4f02-4420-8177-cb955f064b43 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:11.778875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4998ef1a-4f02-4420-8177-cb955f064b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:11.882161 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998ef1a-4f02-4420-8177-cb955f064b43 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:11.921504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:11.955709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1ef63-3180-4ec4-aa45-3dc300f1d722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:11.956356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1ef63-3180-4ec4-aa45-3dc300f1d722 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:11.956667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1ef63-3180-4ec4-aa45-3dc300f1d722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:11.956938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1ef63-3180-4ec4-aa45-3dc300f1d722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:11.958578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ada1ef63-3180-4ec4-aa45-3dc300f1d722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:12.066191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1ef63-3180-4ec4-aa45-3dc300f1d722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:12.067620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1ef63-3180-4ec4-aa45-3dc300f1d722 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:12.085352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 729/1449] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:14.191541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbf051-3430-42f1-bae0-6fe57577e295 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:14.192076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbf051-3430-42f1-bae0-6fe57577e295 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:14.192076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbf051-3430-42f1-bae0-6fe57577e295 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:14.193448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:14.199352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a14d3-699f-426c-a4a5-2d09844bed44 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:14.199809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a14d3-699f-426c-a4a5-2d09844bed44 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:14.199925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a14d3-699f-426c-a4a5-2d09844bed44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:14.265293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-930a14d3-699f-426c-a4a5-2d09844bed44 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:14.266482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-930a14d3-699f-426c-a4a5-2d09844bed44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:14.286290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:14.360918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:14.399181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbf051-3430-42f1-bae0-6fe57577e295 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:14.400772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f6bbf051-3430-42f1-bae0-6fe57577e295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:14.503002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbf051-3430-42f1-bae0-6fe57577e295 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:14.522632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:14.575520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df43bad-8b2b-44d7-98af-0c749b1a4104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:14.576115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df43bad-8b2b-44d7-98af-0c749b1a4104 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:14.576389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df43bad-8b2b-44d7-98af-0c749b1a4104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:14.576689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df43bad-8b2b-44d7-98af-0c749b1a4104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:14.578607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5df43bad-8b2b-44d7-98af-0c749b1a4104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:14.685708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df43bad-8b2b-44d7-98af-0c749b1a4104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:14.687122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df43bad-8b2b-44d7-98af-0c749b1a4104 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:14.705425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 722/1452] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:16.742451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd69be-15d3-4315-8cef-3fbcc9999055 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:16.743018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd69be-15d3-4315-8cef-3fbcc9999055 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:16.743018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd69be-15d3-4315-8cef-3fbcc9999055 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:16.744320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:16.749657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6451c-ebe0-43cc-97fc-ceaebb241e9a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:16.750078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6451c-ebe0-43cc-97fc-ceaebb241e9a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:16.750180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6451c-ebe0-43cc-97fc-ceaebb241e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:16.822574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:16.825189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-36d6451c-ebe0-43cc-97fc-ceaebb241e9a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:16.826383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-36d6451c-ebe0-43cc-97fc-ceaebb241e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:16.918815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:16.957530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd69be-15d3-4315-8cef-3fbcc9999055 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:16.959120 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-61bd69be-15d3-4315-8cef-3fbcc9999055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:17.060643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd69be-15d3-4315-8cef-3fbcc9999055 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:17.115924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:17.140939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d7919-40b5-42ae-9b5f-523be68a25b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:17.141527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d7919-40b5-42ae-9b5f-523be68a25b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:17.141798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d7919-40b5-42ae-9b5f-523be68a25b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:17.142056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d7919-40b5-42ae-9b5f-523be68a25b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:17.143719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c34d7919-40b5-42ae-9b5f-523be68a25b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:17.249510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d7919-40b5-42ae-9b5f-523be68a25b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:17.250943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d7919-40b5-42ae-9b5f-523be68a25b8 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:17.269362 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 732/1455] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:19.336336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb7ddf-d968-42a3-bc40-5e4050f3eba3 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:19.337457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb7ddf-d968-42a3-bc40-5e4050f3eba3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:19.337457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb7ddf-d968-42a3-bc40-5e4050f3eba3 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:19.338843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:19.344655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4702f8-4610-428b-8d89-11d465802f05 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:19.345137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4702f8-4610-428b-8d89-11d465802f05 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:19.345243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4702f8-4610-428b-8d89-11d465802f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:19.418151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6c4702f8-4610-428b-8d89-11d465802f05 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:19.419804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6c4702f8-4610-428b-8d89-11d465802f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:19.452094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:19.519705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:19.559016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb7ddf-d968-42a3-bc40-5e4050f3eba3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:19.560619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cacb7ddf-d968-42a3-bc40-5e4050f3eba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:19.663469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb7ddf-d968-42a3-bc40-5e4050f3eba3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:19.693036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:19.744900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24188c-f3e7-4977-aa7d-e89bc981b389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:19.746202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24188c-f3e7-4977-aa7d-e89bc981b389 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:19.746479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24188c-f3e7-4977-aa7d-e89bc981b389 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:19.746746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24188c-f3e7-4977-aa7d-e89bc981b389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:19.748340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0a24188c-f3e7-4977-aa7d-e89bc981b389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:19.854553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24188c-f3e7-4977-aa7d-e89bc981b389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:19.855979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24188c-f3e7-4977-aa7d-e89bc981b389 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:19.874133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 725/1458] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:21.911070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08f9aa-96eb-4249-8b1e-438b1f58d9fa None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:21.911596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08f9aa-96eb-4249-8b1e-438b1f58d9fa None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:21.911596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08f9aa-96eb-4249-8b1e-438b1f58d9fa None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:21.912921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:21.918038 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a2f36-7532-4023-a5cc-6af844c2a038 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:21.918444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a2f36-7532-4023-a5cc-6af844c2a038 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:21.918577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a2f36-7532-4023-a5cc-6af844c2a038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:21.985291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-eb6a2f36-7532-4023-a5cc-6af844c2a038 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:21.986485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eb6a2f36-7532-4023-a5cc-6af844c2a038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:21.994534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:22.081639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:03:22.121284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08f9aa-96eb-4249-8b1e-438b1f58d9fa None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:22.122819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-df08f9aa-96eb-4249-8b1e-438b1f58d9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:22.225165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08f9aa-96eb-4249-8b1e-438b1f58d9fa None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:22.278747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:22.296316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce78d-77d2-4016-8a40-ccd30e98b6db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:22.296932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce78d-77d2-4016-8a40-ccd30e98b6db service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:22.297207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce78d-77d2-4016-8a40-ccd30e98b6db service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:22.297472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce78d-77d2-4016-8a40-ccd30e98b6db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:22.299186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2afce78d-77d2-4016-8a40-ccd30e98b6db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:22.405136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce78d-77d2-4016-8a40-ccd30e98b6db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:22.406550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce78d-77d2-4016-8a40-ccd30e98b6db service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:22.424210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 735/1461] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:24.475707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b718a43-b479-4bf3-a51a-c35490750774 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:24.476276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b718a43-b479-4bf3-a51a-c35490750774 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:24.476276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b718a43-b479-4bf3-a51a-c35490750774 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:24.478173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:24.483333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3e00d-1286-449d-b299-f1b0569dd83e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:24.483761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3e00d-1286-449d-b299-f1b0569dd83e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:24.483858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3e00d-1286-449d-b299-f1b0569dd83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:24.549216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1ee3e00d-1286-449d-b299-f1b0569dd83e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:24.550430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1ee3e00d-1286-449d-b299-f1b0569dd83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:24.589466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:24.646562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:03:24.684726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b718a43-b479-4bf3-a51a-c35490750774 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:24.686229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0b718a43-b479-4bf3-a51a-c35490750774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:24.787361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b718a43-b479-4bf3-a51a-c35490750774 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:24.825002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:24.861507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce94a3d-4ece-4430-88bc-84e7293860ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:24.862101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce94a3d-4ece-4430-88bc-84e7293860ea service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:24.862374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce94a3d-4ece-4430-88bc-84e7293860ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:24.862639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce94a3d-4ece-4430-88bc-84e7293860ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:24.864220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9ce94a3d-4ece-4430-88bc-84e7293860ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:24.969573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce94a3d-4ece-4430-88bc-84e7293860ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:24.970994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce94a3d-4ece-4430-88bc-84e7293860ea service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:24.988991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 728/1464] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:26.989943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04a41d-62b7-4200-81a1-c2b41a77906c None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:26.990493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04a41d-62b7-4200-81a1-c2b41a77906c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:26.990493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04a41d-62b7-4200-81a1-c2b41a77906c None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:26.991803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:26.996971 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1afb2-2966-4782-8c3d-af43fd85b727 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:26.997376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1afb2-2966-4782-8c3d-af43fd85b727 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:26.997618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1afb2-2966-4782-8c3d-af43fd85b727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:27.063168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1bf1afb2-2966-4782-8c3d-af43fd85b727 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:27.064279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1bf1afb2-2966-4782-8c3d-af43fd85b727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:27.083240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:27.168063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:27.204821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04a41d-62b7-4200-81a1-c2b41a77906c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:27.206378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2c04a41d-62b7-4200-81a1-c2b41a77906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:27.308898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04a41d-62b7-4200-81a1-c2b41a77906c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:27.376249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:27.382684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3c8c3-8314-4319-a8bc-6d9f7d75a018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:27.383777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3c8c3-8314-4319-a8bc-6d9f7d75a018 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:27.384072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3c8c3-8314-4319-a8bc-6d9f7d75a018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:27.384333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3c8c3-8314-4319-a8bc-6d9f7d75a018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:27.385950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a5e3c8c3-8314-4319-a8bc-6d9f7d75a018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:27.492611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3c8c3-8314-4319-a8bc-6d9f7d75a018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:27.494537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3c8c3-8314-4319-a8bc-6d9f7d75a018 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:27.512645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 738/1467] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:29.511440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714375a-908b-47b1-b4ac-9ac70fa5ca97 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:29.512040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714375a-908b-47b1-b4ac-9ac70fa5ca97 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:29.512388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714375a-908b-47b1-b4ac-9ac70fa5ca97 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:29.514318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:03:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:29.521853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecb730-3d97-43b1-a049-3629530b0ac8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:29.522316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecb730-3d97-43b1-a049-3629530b0ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:29.522507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecb730-3d97-43b1-a049-3629530b0ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:29.588560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f6ecb730-3d97-43b1-a049-3629530b0ac8 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:29.589690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f6ecb730-3d97-43b1-a049-3629530b0ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:29.636832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:29.685235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:29.722520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714375a-908b-47b1-b4ac-9ac70fa5ca97 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:29.724075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6714375a-908b-47b1-b4ac-9ac70fa5ca97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:29.828307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714375a-908b-47b1-b4ac-9ac70fa5ca97 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:29.874338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:29.901055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5301fb5-5dd8-4396-92da-de0bbc08bdf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:29.901653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5301fb5-5dd8-4396-92da-de0bbc08bdf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:29.901922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5301fb5-5dd8-4396-92da-de0bbc08bdf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:29.902184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5301fb5-5dd8-4396-92da-de0bbc08bdf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:29.903790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a5301fb5-5dd8-4396-92da-de0bbc08bdf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:30.009100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5301fb5-5dd8-4396-92da-de0bbc08bdf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:30.010553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5301fb5-5dd8-4396-92da-de0bbc08bdf6 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:30.029618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 731/1470] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:39.683494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=22 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:40.027429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=66 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:49.550205 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe548e10-cf63-445d-b2c5-069bba011eb5 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:49.550747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe548e10-cf63-445d-b2c5-069bba011eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:49.550747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe548e10-cf63-445d-b2c5-069bba011eb5 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:49.552080 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 740/1471] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 18:03:49.557871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf3147-c2ae-4e12-a034-35f378491bc7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:49.561477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf3147-c2ae-4e12-a034-35f378491bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:49.561477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf3147-c2ae-4e12-a034-35f378491bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:49.625383 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-73bf3147-c2ae-4e12-a034-35f378491bc7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:49.626616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-73bf3147-c2ae-4e12-a034-35f378491bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:49.724672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 732/1472] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:49.808393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f5a89-75ef-46de-83ca-8ec8e4f3d099 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:49.808704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f5a89-75ef-46de-83ca-8ec8e4f3d099 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:49.808974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f5a89-75ef-46de-83ca-8ec8e4f3d099 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:49.810304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 741/1473] 10.222.0.50 () {54 vars in 967 bytes} [Mon Aug 8 18:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:49.814984 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b336c-e7e2-49c3-82f7-a4f897c92ccf None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:49.815211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b336c-e7e2-49c3-82f7-a4f897c92ccf None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:49.815485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b336c-e7e2-49c3-82f7-a4f897c92ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:49.893699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2e1b336c-e7e2-49c3-82f7-a4f897c92ccf None None] MFA Rules not processed for user `0e95d295337b4472be946d252ac3bd7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:49.894914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2e1b336c-e7e2-49c3-82f7-a4f897c92ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:49.916776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:49.972262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 733/1474] 10.222.0.50 () {58 vars in 1064 bytes} [Mon Aug 8 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:49.980425 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f5a89-75ef-46de-83ca-8ec8e4f3d099 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:49.982027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-153f5a89-75ef-46de-83ca-8ec8e4f3d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:50.081840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f5a89-75ef-46de-83ca-8ec8e4f3d099 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:50.152056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a50f8f-d451-44dd-a227-405e8261cd1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e95d295337b4472be946d252ac3bd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:50.152666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50f8f-d451-44dd-a227-405e8261cd1c service glance] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:50.152942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50f8f-d451-44dd-a227-405e8261cd1c service glance] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:50.153207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50f8f-d451-44dd-a227-405e8261cd1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:50.154815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-37a50f8f-d451-44dd-a227-405e8261cd1c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:50.209939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:50.265557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a50f8f-d451-44dd-a227-405e8261cd1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:50.266995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a50f8f-d451-44dd-a227-405e8261cd1c service glance] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:50.285473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 742/1475] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:52.380497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e1b6e-3e4e-4cab-95dd-2f9a034befcb None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:52.381081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e1b6e-3e4e-4cab-95dd-2f9a034befcb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:52.381081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e1b6e-3e4e-4cab-95dd-2f9a034befcb None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:52.383043 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 734/1476] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:03:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:52.388269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3c2cc-e630-4cff-8fac-0b6b96b38f66 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:52.388734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3c2cc-e630-4cff-8fac-0b6b96b38f66 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:52.388844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3c2cc-e630-4cff-8fac-0b6b96b38f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:52.453610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-04c3c2cc-e630-4cff-8fac-0b6b96b38f66 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:52.454802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-04c3c2cc-e630-4cff-8fac-0b6b96b38f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:52.549487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 743/1477] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:52.572969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e1b6e-3e4e-4cab-95dd-2f9a034befcb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:52.574521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a48e1b6e-3e4e-4cab-95dd-2f9a034befcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:52.698156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e1b6e-3e4e-4cab-95dd-2f9a034befcb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:52.745600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:52.788957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbefe61-9134-448a-9d07-11a7bda7dc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:52.789615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbefe61-9134-448a-9d07-11a7bda7dc51 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:52.789879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbefe61-9134-448a-9d07-11a7bda7dc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:52.790140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbefe61-9134-448a-9d07-11a7bda7dc51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:52.790767 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbefe61-9134-448a-9d07-11a7bda7dc51 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:52.792519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-9cbefe61-9134-448a-9d07-11a7bda7dc51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:03:52.792519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Aug 08 18:03:52.792519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: Aug 08 18:03:52.793648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbefe61-9134-448a-9d07-11a7bda7dc51 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:52.813324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 735/1478] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 18:03:52 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:03:54.830713 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987997e-6929-4209-adf0-685c8ceaad24 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:54.831215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987997e-6929-4209-adf0-685c8ceaad24 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:54.831328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987997e-6929-4209-adf0-685c8ceaad24 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:54.832633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 744/1479] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:54.837879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3980ca-17b0-4b76-a5a0-e6c2ac1e29b0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:54.838311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3980ca-17b0-4b76-a5a0-e6c2ac1e29b0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:54.838419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3980ca-17b0-4b76-a5a0-e6c2ac1e29b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:54.904105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fa3980ca-17b0-4b76-a5a0-e6c2ac1e29b0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:54.905344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fa3980ca-17b0-4b76-a5a0-e6c2ac1e29b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:54.999988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 736/1480] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:55.042410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987997e-6929-4209-adf0-685c8ceaad24 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:55.043969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a987997e-6929-4209-adf0-685c8ceaad24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:55.059391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:55.187918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987997e-6929-4209-adf0-685c8ceaad24 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:55.266146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:55.298058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a67f05-8776-4877-83b1-7c5f4efac8c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:55.298653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a67f05-8776-4877-83b1-7c5f4efac8c7 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:55.298929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a67f05-8776-4877-83b1-7c5f4efac8c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:55.299197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a67f05-8776-4877-83b1-7c5f4efac8c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:55.300802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-57a67f05-8776-4877-83b1-7c5f4efac8c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:55.388725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a67f05-8776-4877-83b1-7c5f4efac8c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:55.390238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a67f05-8776-4877-83b1-7c5f4efac8c7 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:55.407590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 745/1481] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:03:57.592400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd40dc5-05b8-4fae-8ba8-dce4986601f9 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:57.593017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd40dc5-05b8-4fae-8ba8-dce4986601f9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:57.593017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd40dc5-05b8-4fae-8ba8-dce4986601f9 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:57.594299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 737/1482] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:03:57.599385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02940a-83f1-4ac9-b7e2-9180e043186d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:57.599825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02940a-83f1-4ac9-b7e2-9180e043186d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:57.599932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02940a-83f1-4ac9-b7e2-9180e043186d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:57.665785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0e02940a-83f1-4ac9-b7e2-9180e043186d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:03:57.666974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0e02940a-83f1-4ac9-b7e2-9180e043186d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:57.712115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:57.762014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 746/1483] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:03:57.797961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd40dc5-05b8-4fae-8ba8-dce4986601f9 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:03:57.799524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7fd40dc5-05b8-4fae-8ba8-dce4986601f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:57.823187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:57.903045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd40dc5-05b8-4fae-8ba8-dce4986601f9 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:03:57.979358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a283a0-d9fb-4acf-8e76-d20146747d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:03:57.979954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a283a0-d9fb-4acf-8e76-d20146747d67 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:03:57.980227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a283a0-d9fb-4acf-8e76-d20146747d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:03:57.980546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a283a0-d9fb-4acf-8e76-d20146747d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:03:57.982124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-75a283a0-d9fb-4acf-8e76-d20146747d67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:03:58.054360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:03:58.089555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a283a0-d9fb-4acf-8e76-d20146747d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:03:58.090979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a283a0-d9fb-4acf-8e76-d20146747d67 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:03:58.108901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 738/1484] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:04:00.147296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93f64-66fc-493e-88ed-39c7e653a648 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:04:00.147854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93f64-66fc-493e-88ed-39c7e653a648 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:04:00.147854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93f64-66fc-493e-88ed-39c7e653a648 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:04:00.149233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 747/1485] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:04:00.154318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29590029-b2e1-476c-bf4f-f20f4dfc77ea None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:04:00.154767 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29590029-b2e1-476c-bf4f-f20f4dfc77ea None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:04:00.154877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29590029-b2e1-476c-bf4f-f20f4dfc77ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:04:00.221052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-29590029-b2e1-476c-bf4f-f20f4dfc77ea None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:04:00.222255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-29590029-b2e1-476c-bf4f-f20f4dfc77ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:04:00.324691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 739/1486] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:04:00.357520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93f64-66fc-493e-88ed-39c7e653a648 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:04:00.359085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bcc93f64-66fc-493e-88ed-39c7e653a648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:04:00.461874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93f64-66fc-493e-88ed-39c7e653a648 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:04:00.532952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:04:00.534739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2879b-ca96-4963-9d9f-cdb2253ae3bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:04:00.535363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2879b-ca96-4963-9d9f-cdb2253ae3bf service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:04:00.535632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2879b-ca96-4963-9d9f-cdb2253ae3bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:04:00.535900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2879b-ca96-4963-9d9f-cdb2253ae3bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:04:00.537570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3fd2879b-ca96-4963-9d9f-cdb2253ae3bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:04:00.643894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2879b-ca96-4963-9d9f-cdb2253ae3bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:04:00.645319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2879b-ca96-4963-9d9f-cdb2253ae3bf service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:04:00.669000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 748/1487] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:04:02.836369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ab12e-6a9e-4314-a705-14c32c725e00 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:04:02.836943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ab12e-6a9e-4314-a705-14c32c725e00 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:04:02.836943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ab12e-6a9e-4314-a705-14c32c725e00 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:04:02.838250 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 740/1488] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:04:02.843468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c29aab-09ac-457d-9b08-dcf7aecfa0d1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:04:02.843897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c29aab-09ac-457d-9b08-dcf7aecfa0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:04:02.844001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c29aab-09ac-457d-9b08-dcf7aecfa0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:04:02.908688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-65c29aab-09ac-457d-9b08-dcf7aecfa0d1 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:04:02.909890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-65c29aab-09ac-457d-9b08-dcf7aecfa0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:04:02.962413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:04:03.005254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 749/1489] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:04:03.046830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ab12e-6a9e-4314-a705-14c32c725e00 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:04:03.048339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e55ab12e-6a9e-4314-a705-14c32c725e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:04:03.083718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:04:03.151064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ab12e-6a9e-4314-a705-14c32c725e00 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:04:03.222029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95951f4-27ca-4961-b3c4-ed4a50567faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:04:03.222616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95951f4-27ca-4961-b3c4-ed4a50567faa service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:04:03.222892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95951f4-27ca-4961-b3c4-ed4a50567faa service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:04:03.223161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95951f4-27ca-4961-b3c4-ed4a50567faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:04:03.224772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b95951f4-27ca-4961-b3c4-ed4a50567faa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:04:03.302940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:04:03.332825 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95951f4-27ca-4961-b3c4-ed4a50567faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:04:03.334315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95951f4-27ca-4961-b3c4-ed4a50567faa service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:04:03.352449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 741/1490] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:04:05.770977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b1928-161d-4925-921d-f91898e088ca None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:04:05.771535 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b1928-161d-4925-921d-f91898e088ca None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:04:05.771535 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b1928-161d-4925-921d-f91898e088ca None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:04:05.772918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 750/1491] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 18:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:04:05.778683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631b1de-38dd-441b-a21b-fb9626711f7a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:04:05.779086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631b1de-38dd-441b-a21b-fb9626711f7a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:04:05.779240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631b1de-38dd-441b-a21b-fb9626711f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:04:05.845717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f631b1de-38dd-441b-a21b-fb9626711f7a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:04:05.846941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f631b1de-38dd-441b-a21b-fb9626711f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:04:05.947243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 742/1492] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 18:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:04:05.986850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b1928-161d-4925-921d-f91898e088ca None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:04:05.988520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-668b1928-161d-4925-921d-f91898e088ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:04:06.090500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b1928-161d-4925-921d-f91898e088ca None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:04:06.161181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc0180-e24f-4ce3-99b6-15468a0fe676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:04:06.161780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc0180-e24f-4ce3-99b6-15468a0fe676 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:04:06.162051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc0180-e24f-4ce3-99b6-15468a0fe676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:04:06.162316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc0180-e24f-4ce3-99b6-15468a0fe676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:04:06.163923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e0dc0180-e24f-4ce3-99b6-15468a0fe676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:04:06.172810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:04:06.271358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc0180-e24f-4ce3-99b6-15468a0fe676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:04:06.272744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc0180-e24f-4ce3-99b6-15468a0fe676 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:04:06.290756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 751/1493] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:04:15.945780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=82 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:04:16.288601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=50 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:24.822927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916efe-4f82-4459-a27d-0e8894c1aa69 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:24.823394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916efe-4f82-4459-a27d-0e8894c1aa69 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:24.823498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916efe-4f82-4459-a27d-0e8894c1aa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:24.896248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5e916efe-4f82-4459-a27d-0e8894c1aa69 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:05:24.898052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5e916efe-4f82-4459-a27d-0e8894c1aa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:24.990295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 743/1494] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:05:24.998321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62fec2-608e-46a3-96c6-b47c82a0fc38 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:24.998837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62fec2-608e-46a3-96c6-b47c82a0fc38 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:24.998948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62fec2-608e-46a3-96c6-b47c82a0fc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:25.072918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ca62fec2-608e-46a3-96c6-b47c82a0fc38 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:05:25.074033 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ca62fec2-608e-46a3-96c6-b47c82a0fc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:25.168867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 752/1495] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:05:25.173576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e916efe-4f82-4459-a27d-0e8894c1aa69 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:25.174760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5e916efe-4f82-4459-a27d-0e8894c1aa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:25.186490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e916efe-4f82-4459-a27d-0e8894c1aa69 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:25.248345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:25.281115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:25.281707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:25.281979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:25.282246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:25.282881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:05:25.283171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:25.284161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:05:25.284161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Aug 08 18:05:25.284161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: Aug 08 18:05:25.285070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:25.296173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 744/1496] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:05:25.305037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62fec2-608e-46a3-96c6-b47c82a0fc38 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:25.313996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62fec2-608e-46a3-96c6-b47c82a0fc38 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:25.374383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:25.408224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:25.409119 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:25.409390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:25.409652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:25.410254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:25.411973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:05:25.411973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 08 18:05:25.411973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: Aug 08 18:05:25.413184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:25.518174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 753/1497] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:05:25 2022] POST /identity/v3/projects => generated 377 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:05:25.522413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:25.523941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:25.557250 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc74b7-5579-4d5a-a545-dcdcdce0ccae admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:25.647662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:25.648195 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:25.648464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:25.648758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:25.649364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:25.650367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:05:25.650367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Aug 08 18:05:25.650367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: Aug 08 18:05:25.651582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:25.769305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 745/1498] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:05:25 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:05:25.773444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:25.774942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:25.836077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc02f9-ba04-4014-af47-cd5c5c7fa8df admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:25.918162 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:25.928784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:25.929380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:25.929655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:25.929921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:25.930547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:25.931551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:05:25.931551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 08 18:05:25.931551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: Aug 08 18:05:25.932442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:25.944600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 754/1499] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:05:25.948774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:25.950287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:25.962624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf7737-580c-4e08-bcb5-fe23841dce75 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:25.987584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:26.057350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a748a74-e8e2-40c0-93ca-ce9c652ee41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:26.057948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a748a74-e8e2-40c0-93ca-ce9c652ee41c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:26.058224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a748a74-e8e2-40c0-93ca-ce9c652ee41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:26.058490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a748a74-e8e2-40c0-93ca-ce9c652ee41c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:26.059124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a748a74-e8e2-40c0-93ca-ce9c652ee41c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:26.060120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-0a748a74-e8e2-40c0-93ca-ce9c652ee41c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:05:26.060120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Aug 08 18:05:26.060120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: Aug 08 18:05:26.061060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a748a74-e8e2-40c0-93ca-ce9c652ee41c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:26.072967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 746/1500] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:05:26.077478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:26.079014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:26.114027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87c3bc-858a-4b61-8edd-1350f7326bc0 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:26.205210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4dba-dc40-4d31-b6a5-c27db2e805dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:26.205811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4dba-dc40-4d31-b6a5-c27db2e805dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:26.206086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4dba-dc40-4d31-b6a5-c27db2e805dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:26.206352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4dba-dc40-4d31-b6a5-c27db2e805dd admin admin] PATH_INFO: `/v3/projects/cde4f7a78bdf468cb1fd33d27d2541ff/users/5d9b5193ff8d402893d404a9c2f0505f/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:26.228435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e4dba-dc40-4d31-b6a5-c27db2e805dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde4f7a78bdf468cb1fd33d27d2541ff, user_id=5d9b5193ff8d402893d404a9c2f0505f, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:26.229487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-cd2e4dba-dc40-4d31-b6a5-c27db2e805dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:05:26.229487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 08 18:05:26.229487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: Aug 08 18:05:26.231424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e4dba-dc40-4d31-b6a5-c27db2e805dd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:26.331446 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 755/1501] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:05:26 2022] PUT /identity/v3/projects/cde4f7a78bdf468cb1fd33d27d2541ff/users/5d9b5193ff8d402893d404a9c2f0505f/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:05:26.344130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4b91d-e04c-40bc-a511-dee3771380c3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:26.344425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4b91d-e04c-40bc-a511-dee3771380c3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:26.344719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4b91d-e04c-40bc-a511-dee3771380c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:26.400962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-99c4b91d-e04c-40bc-a511-dee3771380c3 None None] MFA Rules not processed for user `5d9b5193ff8d402893d404a9c2f0505f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:05:26.402166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-99c4b91d-e04c-40bc-a511-dee3771380c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:26.477182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 747/1502] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:05:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:05:26.490207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a617a9-215a-4b3e-bcf7-b7b52fe682bb None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:26.490485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a617a9-215a-4b3e-bcf7-b7b52fe682bb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:26.490752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a617a9-215a-4b3e-bcf7-b7b52fe682bb None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:26.492077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 756/1503] 10.222.0.50 () {54 vars in 964 bytes} [Mon Aug 8 18:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:05:26.496750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfea2e5-b7ba-416c-8898-0b9a8d3da3dd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:26.497061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfea2e5-b7ba-416c-8898-0b9a8d3da3dd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:26.497335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfea2e5-b7ba-416c-8898-0b9a8d3da3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:26.528776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:26.571970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7cfea2e5-b7ba-416c-8898-0b9a8d3da3dd None None] MFA Rules not processed for user `5b4aef937fb047f6869a156a68e7bf2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:05:26.573255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7cfea2e5-b7ba-416c-8898-0b9a8d3da3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:26.647753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 748/1504] 10.222.0.50 () {58 vars in 1061 bytes} [Mon Aug 8 18:05:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:05:26.655411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a617a9-215a-4b3e-bcf7-b7b52fe682bb None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:26.656950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-28a617a9-215a-4b3e-bcf7-b7b52fe682bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:26.706302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:26.759296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a617a9-215a-4b3e-bcf7-b7b52fe682bb None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:26.832020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:26.832651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:26.832926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:26.833191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:26.834769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:26.931234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:26.932695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:26.936147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:26.953693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 757/1505] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:05:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:05:27.000357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebfe77-ceeb-4b2a-aacd-4ab73d44d0e0 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:27.000818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebfe77-ceeb-4b2a-aacd-4ab73d44d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:27.000954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebfe77-ceeb-4b2a-aacd-4ab73d44d0e0 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:27.002228 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 749/1506] 10.222.0.50 () {52 vars in 830 bytes} [Mon Aug 8 18:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:05:27.005817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:27.007034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:27.105138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02fc7a-1b08-43bc-9b86-76fab8884564 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:27.198088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:27.198712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:27.198986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:27.199251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] PATH_INFO: `/v3/users/5d9b5193ff8d402893d404a9c2f0505f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:27.211787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9b5193ff8d402893d404a9c2f0505f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:27.212844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:05:27.212844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 08 18:05:27.212844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: Aug 08 18:05:27.214056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:27.223651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:27.372497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9b5193ff8d402893d404a9c2f0505f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:27.437978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9b5193ff8d402893d404a9c2f0505f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:27.451988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9b5193ff8d402893d404a9c2f0505f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:27.452529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9b5193ff8d402893d404a9c2f0505f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:27.452968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9b5193ff8d402893d404a9c2f0505f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:27.481574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 758/1507] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:05:27 2022] DELETE /identity/v3/users/5d9b5193ff8d402893d404a9c2f0505f => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:05:27.485966 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a3f59-3458-4aa9-b7ef-0304074a1987 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:27.486245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a3f59-3458-4aa9-b7ef-0304074a1987 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:27.486512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a3f59-3458-4aa9-b7ef-0304074a1987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:27.543040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cb2a3f59-3458-4aa9-b7ef-0304074a1987 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:05:27.544243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cb2a3f59-3458-4aa9-b7ef-0304074a1987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:27.660866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:27.674137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 750/1508] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:05:27.683612 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:27.684864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:27.784916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d8290-e4de-4285-a8ac-fca672105c78 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:27.838476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:27.858348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90056b-dbfd-48e6-9ecb-34a81b08211f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:27.858955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90056b-dbfd-48e6-9ecb-34a81b08211f service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:27.859227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90056b-dbfd-48e6-9ecb-34a81b08211f service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:27.859497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90056b-dbfd-48e6-9ecb-34a81b08211f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:27.861186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5a90056b-dbfd-48e6-9ecb-34a81b08211f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:27.970922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90056b-dbfd-48e6-9ecb-34a81b08211f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:27.972295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90056b-dbfd-48e6-9ecb-34a81b08211f service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:27.992000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 759/1509] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:05:28.411451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a3f59-3458-4aa9-b7ef-0304074a1987 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:05:28.413339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cb2a3f59-3458-4aa9-b7ef-0304074a1987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:05:28.433548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a3f59-3458-4aa9-b7ef-0304074a1987 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:05:28.527503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:05:28.528035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:05:28.528313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:05:28.528606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] PATH_INFO: `/v3/projects/cde4f7a78bdf468cb1fd33d27d2541ff` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:05:28.538633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde4f7a78bdf468cb1fd33d27d2541ff)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:05:28.542991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:05:28.542991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Aug 08 18:05:28.542991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: Aug 08 18:05:28.542991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:05:28.648397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde4f7a78bdf468cb1fd33d27d2541ff', 'request_id': 'req-de0b12fc-cfc7-4711-bec3-df02ed4c186a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:28.654329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde4f7a78bdf468cb1fd33d27d2541ff', 'request_id': 'req-de0b12fc-cfc7-4711-bec3-df02ed4c186a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:28.723629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde4f7a78bdf468cb1fd33d27d2541ff', 'request_id': 'req-de0b12fc-cfc7-4711-bec3-df02ed4c186a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:28.753032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] The token cache is being invalidate because project cde4f7a78bdf468cb1fd33d27d2541ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:05:28.753315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:28.753830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-de0b12fc-cfc7-4711-bec3-df02ed4c186a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:05:28.755131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 751/1510] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:05:28 2022] DELETE /identity/v3/projects/cde4f7a78bdf468cb1fd33d27d2541ff => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:05:37.989709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=62 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:05:38.751091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:06:56.615626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc13208-098c-4c24-ab75-13f283c03dfa None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:06:56.616240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc13208-098c-4c24-ab75-13f283c03dfa None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:06:56.616240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc13208-098c-4c24-ab75-13f283c03dfa None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:06:56.617571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:06:56.622447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99daa3c-3717-4d12-bea9-94a0caa7e7da None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:06:56.622886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99daa3c-3717-4d12-bea9-94a0caa7e7da None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:06:56.623005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99daa3c-3717-4d12-bea9-94a0caa7e7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:06:56.702609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a99daa3c-3717-4d12-bea9-94a0caa7e7da None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:06:56.703807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a99daa3c-3717-4d12-bea9-94a0caa7e7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:06:56.807285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:06:56.960777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc13208-098c-4c24-ab75-13f283c03dfa None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:06:56.964843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0fc13208-098c-4c24-ab75-13f283c03dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:06:57.088791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc13208-098c-4c24-ab75-13f283c03dfa None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:06:57.094929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d688c-9b3e-46a6-9815-51a9722b7318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:06:57.095536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d688c-9b3e-46a6-9815-51a9722b7318 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:06:57.095809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d688c-9b3e-46a6-9815-51a9722b7318 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:06:57.096077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d688c-9b3e-46a6-9815-51a9722b7318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:06:57.104567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d688c-9b3e-46a6-9815-51a9722b7318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:06:57.108196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d688c-9b3e-46a6-9815-51a9722b7318 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:06:57.115856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 761/1513] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:06:57.940345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3f40f-a416-4f1e-b143-bf1db7f1b1fc None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:06:57.940909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3f40f-a416-4f1e-b143-bf1db7f1b1fc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:06:57.940909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3f40f-a416-4f1e-b143-bf1db7f1b1fc None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:06:57.942328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:06:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:06:57.947077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24e5e2-cb51-4315-8ce8-fc014d47865e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:06:57.947454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24e5e2-cb51-4315-8ce8-fc014d47865e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:06:57.947610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24e5e2-cb51-4315-8ce8-fc014d47865e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:06:57.967536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3f24e5e2-cb51-4315-8ce8-fc014d47865e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:06:57.968767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3f24e5e2-cb51-4315-8ce8-fc014d47865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:06:57.977268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:06:58.129308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3f40f-a416-4f1e-b143-bf1db7f1b1fc None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:06:58.143527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3f40f-a416-4f1e-b143-bf1db7f1b1fc None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:06:58.149428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda4b8a-e23d-4bbb-b342-d91f47231578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:06:58.150027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda4b8a-e23d-4bbb-b342-d91f47231578 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:06:58.150301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda4b8a-e23d-4bbb-b342-d91f47231578 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:06:58.150567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda4b8a-e23d-4bbb-b342-d91f47231578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:06:58.158688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda4b8a-e23d-4bbb-b342-d91f47231578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:06:58.160331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda4b8a-e23d-4bbb-b342-d91f47231578 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:06:58.169595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 754/1516] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:06:58.631125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b00119-7a42-4fcc-b2a3-6c02570e4acb None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:06:58.631390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b00119-7a42-4fcc-b2a3-6c02570e4acb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:06:58.631661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b00119-7a42-4fcc-b2a3-6c02570e4acb None None] PATH_INFO: `/v3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:06:58.632986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 763/1517] 10.222.0.72 () {54 vars in 925 bytes} [Mon Aug 8 18:06:58 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:06:58.640227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e1c54-ddbb-47f1-9277-02f422a1a2f3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:06:58.640646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e1c54-ddbb-47f1-9277-02f422a1a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:06:58.641041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e1c54-ddbb-47f1-9277-02f422a1a2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:06:58.699206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fc7e1c54-ddbb-47f1-9277-02f422a1a2f3 None None] MFA Rules not processed for user `b81bdfeedcba4b0c93f13d14318b2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:06:58.700392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fc7e1c54-ddbb-47f1-9277-02f422a1a2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:06:58.717139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:06:58.770370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 755/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Aug 8 18:06:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:07:07.922275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a1277-072c-468c-b9d3-27d2157adbc2 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:07.922865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a1277-072c-468c-b9d3-27d2157adbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:07.922865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a1277-072c-468c-b9d3-27d2157adbc2 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:07.924177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 764/1519] 10.222.0.50 () {54 vars in 961 bytes} [Mon Aug 8 18:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:07:07.929359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b8801-781c-4d5a-a096-d58a35a7339c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:07.932257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b8801-781c-4d5a-a096-d58a35a7339c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:07.932257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b8801-781c-4d5a-a096-d58a35a7339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:07.949746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-850b8801-781c-4d5a-a096-d58a35a7339c None None] MFA Rules not processed for user `b81bdfeedcba4b0c93f13d14318b2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:07:07.951191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-850b8801-781c-4d5a-a096-d58a35a7339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:07.960628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 756/1520] 10.222.0.50 () {58 vars in 1058 bytes} [Mon Aug 8 18:07:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:07:07.962598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=56 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:07:07.968154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a1277-072c-468c-b9d3-27d2157adbc2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:07:07.982196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a1277-072c-468c-b9d3-27d2157adbc2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:07:07.987255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7f2a9-bfee-4a9e-8fb3-359af0fb1d6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:07:07.987836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7f2a9-bfee-4a9e-8fb3-359af0fb1d6a service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:07.988111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7f2a9-bfee-4a9e-8fb3-359af0fb1d6a service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:07.988383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7f2a9-bfee-4a9e-8fb3-359af0fb1d6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:07.996483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7f2a9-bfee-4a9e-8fb3-359af0fb1d6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:07:07.998069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7f2a9-bfee-4a9e-8fb3-359af0fb1d6a service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:07:08.007896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 765/1521] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:07:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:07:09.399652 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9513-97f2-42ff-a4ef-89c22c87ae73 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:09.400304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9513-97f2-42ff-a4ef-89c22c87ae73 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:09.400304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9513-97f2-42ff-a4ef-89c22c87ae73 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:09.401774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:07:09.406693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc84b4-038c-4ebc-aae9-c57fb74afbae None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:09.407113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc84b4-038c-4ebc-aae9-c57fb74afbae None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:09.407264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc84b4-038c-4ebc-aae9-c57fb74afbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:09.426748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-54cc84b4-038c-4ebc-aae9-c57fb74afbae None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:07:09.427931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-54cc84b4-038c-4ebc-aae9-c57fb74afbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:09.435734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:07:09.590152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b9513-97f2-42ff-a4ef-89c22c87ae73 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:07:09.605194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b9513-97f2-42ff-a4ef-89c22c87ae73 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:07:09.610775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95655308-5fcb-4118-ae0b-02df3577733c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:07:09.614437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95655308-5fcb-4118-ae0b-02df3577733c service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:09.614437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95655308-5fcb-4118-ae0b-02df3577733c service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:09.614437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95655308-5fcb-4118-ae0b-02df3577733c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:09.620015 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95655308-5fcb-4118-ae0b-02df3577733c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:07:09.621607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95655308-5fcb-4118-ae0b-02df3577733c service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:07:09.631048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 758/1524] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:07:10.451888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6e7f3-51e2-46f3-8701-99894516ece1 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:10.452432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6e7f3-51e2-46f3-8701-99894516ece1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:10.452889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6e7f3-51e2-46f3-8701-99894516ece1 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:10.455623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 18:07:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:07:10.460960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d3681-f1fe-4603-89ef-e9511a581d09 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:10.461503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d3681-f1fe-4603-89ef-e9511a581d09 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:10.461641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d3681-f1fe-4603-89ef-e9511a581d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:10.481143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-826d3681-f1fe-4603-89ef-e9511a581d09 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:07:10.482330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-826d3681-f1fe-4603-89ef-e9511a581d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:10.490085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 18:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:07:10.641192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6e7f3-51e2-46f3-8701-99894516ece1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:07:10.655792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6e7f3-51e2-46f3-8701-99894516ece1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:07:10.661229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543578b-2e31-489b-8332-945371d5145f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:07:10.661811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543578b-2e31-489b-8332-945371d5145f service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:10.662082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543578b-2e31-489b-8332-945371d5145f service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:10.662347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543578b-2e31-489b-8332-945371d5145f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:10.670142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543578b-2e31-489b-8332-945371d5145f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:07:10.671703 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543578b-2e31-489b-8332-945371d5145f service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:07:10.681388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 768/1527] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:07:14.566631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70a684-27ad-4954-986c-89f1e2a62265 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:14.567202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70a684-27ad-4954-986c-89f1e2a62265 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:14.567202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70a684-27ad-4954-986c-89f1e2a62265 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:14.568549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:07:14.573596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9647e-4000-40ae-921a-999eefdc65bc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:14.573978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9647e-4000-40ae-921a-999eefdc65bc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:14.574130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9647e-4000-40ae-921a-999eefdc65bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:14.593611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c2d9647e-4000-40ae-921a-999eefdc65bc None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:07:14.594834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c2d9647e-4000-40ae-921a-999eefdc65bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:14.602417 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:07:14.674268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70a684-27ad-4954-986c-89f1e2a62265 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:07:14.675985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ed70a684-27ad-4954-986c-89f1e2a62265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:14.753539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70a684-27ad-4954-986c-89f1e2a62265 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:07:14.757082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28bdfd-dde9-4742-8a3a-c2f49a25f0b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:07:14.757684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28bdfd-dde9-4742-8a3a-c2f49a25f0b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:14.757957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28bdfd-dde9-4742-8a3a-c2f49a25f0b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:14.758224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28bdfd-dde9-4742-8a3a-c2f49a25f0b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:14.767715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28bdfd-dde9-4742-8a3a-c2f49a25f0b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:07:14.769174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28bdfd-dde9-4742-8a3a-c2f49a25f0b0 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:07:14.778370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 761/1530] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:07:16.768387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53a955-817d-4e7b-908b-54822a849986 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:16.768997 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53a955-817d-4e7b-908b-54822a849986 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:16.768997 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53a955-817d-4e7b-908b-54822a849986 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:16.770307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:07:16.775369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a6f1d-2a8d-4351-82b3-afa72a1453e4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:16.775790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a6f1d-2a8d-4351-82b3-afa72a1453e4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:16.775938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a6f1d-2a8d-4351-82b3-afa72a1453e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:16.795353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-541a6f1d-2a8d-4351-82b3-afa72a1453e4 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:07:16.796575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-541a6f1d-2a8d-4351-82b3-afa72a1453e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:16.804811 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:07:16.841453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53a955-817d-4e7b-908b-54822a849986 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:07:16.852068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53a955-817d-4e7b-908b-54822a849986 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:07:16.855805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db6456-4e5b-4800-97b3-e11cf6bac991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:07:16.856390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db6456-4e5b-4800-97b3-e11cf6bac991 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:16.856690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db6456-4e5b-4800-97b3-e11cf6bac991 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:16.856956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db6456-4e5b-4800-97b3-e11cf6bac991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:16.866696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db6456-4e5b-4800-97b3-e11cf6bac991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:07:16.868142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db6456-4e5b-4800-97b3-e11cf6bac991 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:07:16.877926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 771/1533] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:07:20.555102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c29c6-3b26-4a0d-b243-4c7563eb22b5 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:20.555646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c29c6-3b26-4a0d-b243-4c7563eb22b5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:20.555646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c29c6-3b26-4a0d-b243-4c7563eb22b5 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:20.557014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:07:20.562098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8cd46-1fee-4dbb-a154-e0689352b5a7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:20.562523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8cd46-1fee-4dbb-a154-e0689352b5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:20.562667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8cd46-1fee-4dbb-a154-e0689352b5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:20.583210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-92d8cd46-1fee-4dbb-a154-e0689352b5a7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:07:20.584373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-92d8cd46-1fee-4dbb-a154-e0689352b5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:20.592063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:07:20.900087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c29c6-3b26-4a0d-b243-4c7563eb22b5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:07:20.901756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b78c29c6-3b26-4a0d-b243-4c7563eb22b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:20.964980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:07:20.978137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c29c6-3b26-4a0d-b243-4c7563eb22b5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:07:20.981708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca981f-819c-40c3-8bd7-ddf58a4b07b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:07:20.982290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca981f-819c-40c3-8bd7-ddf58a4b07b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:20.982582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca981f-819c-40c3-8bd7-ddf58a4b07b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:20.982843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca981f-819c-40c3-8bd7-ddf58a4b07b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:20.992191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca981f-819c-40c3-8bd7-ddf58a4b07b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:07:20.993706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca981f-819c-40c3-8bd7-ddf58a4b07b1 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:07:21.003346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 764/1536] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:07:23.033798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba834-e6d6-47a0-b60d-298ae80593aa None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:23.034355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba834-e6d6-47a0-b60d-298ae80593aa None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:23.034355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba834-e6d6-47a0-b60d-298ae80593aa None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:23.035719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 18:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:07:23.040764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79157a-23cb-4d0e-ab73-76a9ef69990b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:23.041206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79157a-23cb-4d0e-ab73-76a9ef69990b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:23.041448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79157a-23cb-4d0e-ab73-76a9ef69990b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:23.060552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ff79157a-23cb-4d0e-ab73-76a9ef69990b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:07:23.061663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ff79157a-23cb-4d0e-ab73-76a9ef69990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:23.069447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 18:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:07:23.374960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba834-e6d6-47a0-b60d-298ae80593aa None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:07:23.377203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7d0ba834-e6d6-47a0-b60d-298ae80593aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:07:23.389912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba834-e6d6-47a0-b60d-298ae80593aa None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:07:23.393958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329bb56-6c88-4050-8066-11901378723c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:07:23.394529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329bb56-6c88-4050-8066-11901378723c service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:07:23.394799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329bb56-6c88-4050-8066-11901378723c service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:07:23.395063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329bb56-6c88-4050-8066-11901378723c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:07:23.404155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329bb56-6c88-4050-8066-11901378723c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:07:23.405624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329bb56-6c88-4050-8066-11901378723c service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:07:23.414750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 774/1539] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:07:33.055429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=11 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:07:33.410760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=33 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:46.232148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3cb21-13cb-4e17-a522-b0e2ea9f8483 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.233185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3cb21-13cb-4e17-a522-b0e2ea9f8483 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.233623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3cb21-13cb-4e17-a522-b0e2ea9f8483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.264471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-18a3cb21-13cb-4e17-a522-b0e2ea9f8483 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:46.265709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-18a3cb21-13cb-4e17-a522-b0e2ea9f8483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:46.274650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 766/1540] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:08:46.284239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd95f8-1007-4040-a23f-4b7237b62e8f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.285311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd95f8-1007-4040-a23f-4b7237b62e8f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.285809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd95f8-1007-4040-a23f-4b7237b62e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.322246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-38fd95f8-1007-4040-a23f-4b7237b62e8f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:46.323835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-38fd95f8-1007-4040-a23f-4b7237b62e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:46.337942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 775/1541] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:46.343086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3cb21-13cb-4e17-a522-b0e2ea9f8483 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:46.357777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3cb21-13cb-4e17-a522-b0e2ea9f8483 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:46.362981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b91b7-592a-4585-bc5b-db99db0fd61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:46.363772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b91b7-592a-4585-bc5b-db99db0fd61f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.364209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b91b7-592a-4585-bc5b-db99db0fd61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.364673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b91b7-592a-4585-bc5b-db99db0fd61f admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.365474 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b91b7-592a-4585-bc5b-db99db0fd61f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:08:46.365932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b91b7-592a-4585-bc5b-db99db0fd61f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:46.367781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b91b7-592a-4585-bc5b-db99db0fd61f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:46.381539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 767/1542] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:08:46.389432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd95f8-1007-4040-a23f-4b7237b62e8f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:46.401531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd95f8-1007-4040-a23f-4b7237b62e8f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:46.410741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba048d53-92d9-4439-957d-92a174cb3fb2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.411128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0179-37ac-4ba0-8277-e0b911162fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:46.411561 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba048d53-92d9-4439-957d-92a174cb3fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.411756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0179-37ac-4ba0-8277-e0b911162fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.412002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0179-37ac-4ba0-8277-e0b911162fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.412271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0179-37ac-4ba0-8277-e0b911162fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.412677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba048d53-92d9-4439-957d-92a174cb3fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.412931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b0179-37ac-4ba0-8277-e0b911162fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:46.414844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b0179-37ac-4ba0-8277-e0b911162fdf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:46.441728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ba048d53-92d9-4439-957d-92a174cb3fb2 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:46.443312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ba048d53-92d9-4439-957d-92a174cb3fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:46.461671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 768/1543] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:08:46.472306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565f3cf-92bd-43c7-8484-90bfdb7b6156 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.473183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565f3cf-92bd-43c7-8484-90bfdb7b6156 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.473673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565f3cf-92bd-43c7-8484-90bfdb7b6156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.504604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9565f3cf-92bd-43c7-8484-90bfdb7b6156 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:46.506422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9565f3cf-92bd-43c7-8484-90bfdb7b6156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:46.518542 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 769/1544] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:46.523376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565f3cf-92bd-43c7-8484-90bfdb7b6156 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:46.534560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565f3cf-92bd-43c7-8484-90bfdb7b6156 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:46.542697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca8e7e-292c-4a02-9ed2-a77776f48b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:46.544625 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca8e7e-292c-4a02-9ed2-a77776f48b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.545138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca8e7e-292c-4a02-9ed2-a77776f48b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.545733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca8e7e-292c-4a02-9ed2-a77776f48b18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.546663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca8e7e-292c-4a02-9ed2-a77776f48b18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:08:46.547195 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca8e7e-292c-4a02-9ed2-a77776f48b18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:46.549095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca8e7e-292c-4a02-9ed2-a77776f48b18 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:46.567607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 770/1545] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:08:46.576535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca8e7e-292c-4a02-9ed2-a77776f48b18 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:46.587666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca8e7e-292c-4a02-9ed2-a77776f48b18 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:46.596386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 776/1546] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:08:46.596740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54dbca-6926-4416-af61-17327c45e298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:46.597629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54dbca-6926-4416-af61-17327c45e298 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.598122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54dbca-6926-4416-af61-17327c45e298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.598598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54dbca-6926-4416-af61-17327c45e298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.599507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54dbca-6926-4416-af61-17327c45e298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:46.600752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0179-37ac-4ba0-8277-e0b911162fdf admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:46.601074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-fc54dbca-6926-4416-af61-17327c45e298 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:08:46.601074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Aug 08 18:08:46.601074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: Aug 08 18:08:46.602681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54dbca-6926-4416-af61-17327c45e298 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:46.713206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0179-37ac-4ba0-8277-e0b911162fdf admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:46.719447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 771/1547] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:08:46.723923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3787e8c-2829-44d1-bbe0-dd33c624b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:46.723923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787e8c-2829-44d1-bbe0-dd33c624b9bb admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.723923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787e8c-2829-44d1-bbe0-dd33c624b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.723923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787e8c-2829-44d1-bbe0-dd33c624b9bb admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.724292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3787e8c-2829-44d1-bbe0-dd33c624b9bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:46.724382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54dbca-6926-4416-af61-17327c45e298 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:46.724460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-e3787e8c-2829-44d1-bbe0-dd33c624b9bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:08:46.724460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 08 18:08:46.724460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: Aug 08 18:08:46.725368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3787e8c-2829-44d1-bbe0-dd33c624b9bb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:46.807431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 777/1548] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:46.808843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d9e31-b026-4539-ac08-a693bc313fe3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.809134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d9e31-b026-4539-ac08-a693bc313fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.809404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d9e31-b026-4539-ac08-a693bc313fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.859370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54dbca-6926-4416-af61-17327c45e298 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:46.867925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-5b2d9e31-b026-4539-ac08-a693bc313fe3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:46.869238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5b2d9e31-b026-4539-ac08-a693bc313fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:46.874362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced0c68-51fb-4ea4-b34c-dead1d19f199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:46.875005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced0c68-51fb-4ea4-b34c-dead1d19f199 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.875351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced0c68-51fb-4ea4-b34c-dead1d19f199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.875627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced0c68-51fb-4ea4-b34c-dead1d19f199 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.878009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced0c68-51fb-4ea4-b34c-dead1d19f199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:46.879952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced0c68-51fb-4ea4-b34c-dead1d19f199 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:46.902217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 778/1549] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:46.904090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d9e31-b026-4539-ac08-a693bc313fe3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:46.915417 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d9e31-b026-4539-ac08-a693bc313fe3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:46.920825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e885ff2-eaee-489e-9bef-349b6df8be9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:46.921746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e885ff2-eaee-489e-9bef-349b6df8be9b admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.922286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e885ff2-eaee-489e-9bef-349b6df8be9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.922960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e885ff2-eaee-489e-9bef-349b6df8be9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.923801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e885ff2-eaee-489e-9bef-349b6df8be9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:46.925827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e885ff2-eaee-489e-9bef-349b6df8be9b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:46.938730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 779/1550] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:46.940608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990cf61-0f8d-45d4-b86e-934d0475917b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.941074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990cf61-0f8d-45d4-b86e-934d0475917b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.941596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990cf61-0f8d-45d4-b86e-934d0475917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.965091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e990cf61-0f8d-45d4-b86e-934d0475917b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:46.967151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e990cf61-0f8d-45d4-b86e-934d0475917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:46.974217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 772/1551] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:08:46.974217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe5cf2-ceef-443c-9015-48382fbc3175 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.974217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe5cf2-ceef-443c-9015-48382fbc3175 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.974217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe5cf2-ceef-443c-9015-48382fbc3175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:46.984180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 780/1552] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:46.984883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed700c99-7ed0-4fbb-a829-258da0dd6ed8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:46.985163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed700c99-7ed0-4fbb-a829-258da0dd6ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:46.985472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed700c99-7ed0-4fbb-a829-258da0dd6ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.002341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-52fe5cf2-ceef-443c-9015-48382fbc3175 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.003590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-52fe5cf2-ceef-443c-9015-48382fbc3175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.007894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ed700c99-7ed0-4fbb-a829-258da0dd6ed8 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.009678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ed700c99-7ed0-4fbb-a829-258da0dd6ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.013016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 773/1553] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.013955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe5cf2-ceef-443c-9015-48382fbc3175 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.023596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 781/1554] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.024484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe5cf2-ceef-443c-9015-48382fbc3175 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.024645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed700c99-7ed0-4fbb-a829-258da0dd6ed8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.037130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed700c99-7ed0-4fbb-a829-258da0dd6ed8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.038068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fca2b3-92f0-4368-9819-44d46ea58f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.038686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fca2b3-92f0-4368-9819-44d46ea58f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.038954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fca2b3-92f0-4368-9819-44d46ea58f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.039219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fca2b3-92f0-4368-9819-44d46ea58f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.039864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fca2b3-92f0-4368-9819-44d46ea58f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.041483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fca2b3-92f0-4368-9819-44d46ea58f1f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.042286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d428b-211f-4e8b-862d-19987f18a4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.042887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d428b-211f-4e8b-862d-19987f18a4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.043179 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d428b-211f-4e8b-862d-19987f18a4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.043490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d428b-211f-4e8b-862d-19987f18a4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.044154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d428b-211f-4e8b-862d-19987f18a4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.045780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d428b-211f-4e8b-862d-19987f18a4c6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.052347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 774/1555] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:47.053398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43104847-6cca-4251-b64f-864413be5bcd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.053682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43104847-6cca-4251-b64f-864413be5bcd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.053971 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43104847-6cca-4251-b64f-864413be5bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.059331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 782/1556] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:47.061259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f41a2-d020-4e85-a5e9-0f2c93904d25 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.062035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f41a2-d020-4e85-a5e9-0f2c93904d25 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.063610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f41a2-d020-4e85-a5e9-0f2c93904d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.073947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-43104847-6cca-4251-b64f-864413be5bcd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.075889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-43104847-6cca-4251-b64f-864413be5bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.088540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 775/1557] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.088727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-df7f41a2-d020-4e85-a5e9-0f2c93904d25 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.088837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907efd5-cad0-4f0d-af1d-9f44ee9675b1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.088956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-df7f41a2-d020-4e85-a5e9-0f2c93904d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.089094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907efd5-cad0-4f0d-af1d-9f44ee9675b1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.089094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907efd5-cad0-4f0d-af1d-9f44ee9675b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.103133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 783/1558] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.107195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f41a2-d020-4e85-a5e9-0f2c93904d25 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.114479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6907efd5-cad0-4f0d-af1d-9f44ee9675b1 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.115347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6907efd5-cad0-4f0d-af1d-9f44ee9675b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.118371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f41a2-d020-4e85-a5e9-0f2c93904d25 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.123483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 776/1559] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.123754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.124811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.124941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304875-d0dc-4c2d-a10c-8137f0ba316f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.125112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304875-d0dc-4c2d-a10c-8137f0ba316f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.125358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304875-d0dc-4c2d-a10c-8137f0ba316f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.125904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.126393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.127322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:08:47.127846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.129105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:08:47.129105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 08 18:08:47.129105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: Aug 08 18:08:47.130457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.145305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 784/1560] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:08:47.145944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4d304875-d0dc-4c2d-a10c-8137f0ba316f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.146634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.147270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4d304875-d0dc-4c2d-a10c-8137f0ba316f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.160174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 777/1561] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.161257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d304875-d0dc-4c2d-a10c-8137f0ba316f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.170802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b9b3e-d045-4e71-890c-a994dbb48826 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.173364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d304875-d0dc-4c2d-a10c-8137f0ba316f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.176320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f6161-586d-47b1-865d-de771a1692b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.177074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f6161-586d-47b1-865d-de771a1692b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.177312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f6161-586d-47b1-865d-de771a1692b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.177580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f6161-586d-47b1-865d-de771a1692b7 admin admin] PATH_INFO: `/v3/projects/7b16538b6545458aa54f1edd7f5c5737/users/ec40d2360ab2417bb2d50414575d59d0/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.178435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcc27e-6b43-45cb-ac26-2bfef6aba402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.179041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcc27e-6b43-45cb-ac26-2bfef6aba402 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.179317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcc27e-6b43-45cb-ac26-2bfef6aba402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.179582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcc27e-6b43-45cb-ac26-2bfef6aba402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.180246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dcc27e-6b43-45cb-ac26-2bfef6aba402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.181889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dcc27e-6b43-45cb-ac26-2bfef6aba402 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.192421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 778/1562] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:47.194444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f6161-586d-47b1-865d-de771a1692b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b16538b6545458aa54f1edd7f5c5737, user_id=ec40d2360ab2417bb2d50414575d59d0, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.197044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c872e-2754-485c-a5d4-902b7324ac78 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.197234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c872e-2754-485c-a5d4-902b7324ac78 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.197500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c872e-2754-485c-a5d4-902b7324ac78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.197617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f6161-586d-47b1-865d-de771a1692b7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.218001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-770c872e-2754-485c-a5d4-902b7324ac78 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.219191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-770c872e-2754-485c-a5d4-902b7324ac78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.227145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 779/1563] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.227981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c872e-2754-485c-a5d4-902b7324ac78 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.237681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c872e-2754-485c-a5d4-902b7324ac78 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.242373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89d4cb-3759-4d4a-a55c-434ac2763949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.243061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89d4cb-3759-4d4a-a55c-434ac2763949 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.243411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89d4cb-3759-4d4a-a55c-434ac2763949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.243752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89d4cb-3759-4d4a-a55c-434ac2763949 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.244456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89d4cb-3759-4d4a-a55c-434ac2763949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:08:47.244881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89d4cb-3759-4d4a-a55c-434ac2763949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.246975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89d4cb-3759-4d4a-a55c-434ac2763949 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.258458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 780/1564] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:08:47.259254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89d4cb-3759-4d4a-a55c-434ac2763949 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.270806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89d4cb-3759-4d4a-a55c-434ac2763949 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.273234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 785/1565] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:47 2022] PUT /identity/v3/projects/7b16538b6545458aa54f1edd7f5c5737/users/ec40d2360ab2417bb2d50414575d59d0/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:47.276625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f6161-586d-47b1-865d-de771a1692b7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.280723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:47.310257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:47.331151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133924f-b8a5-4d65-9e61-af5e59a5a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.331834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133924f-b8a5-4d65-9e61-af5e59a5a395 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.332111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133924f-b8a5-4d65-9e61-af5e59a5a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.332377 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133924f-b8a5-4d65-9e61-af5e59a5a395 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.333047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133924f-b8a5-4d65-9e61-af5e59a5a395 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:08:47.333344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133924f-b8a5-4d65-9e61-af5e59a5a395 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.334964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133924f-b8a5-4d65-9e61-af5e59a5a395 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.347254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f6161-586d-47b1-865d-de771a1692b7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.354913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 781/1566] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:08:47.354913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39d1ca-5c5c-448d-b3dd-bc64606fbc33 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.354913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39d1ca-5c5c-448d-b3dd-bc64606fbc33 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.354913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39d1ca-5c5c-448d-b3dd-bc64606fbc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.355304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f49bf-fc6d-44ba-a097-dcc0ef0388d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.355304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f49bf-fc6d-44ba-a097-dcc0ef0388d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.355304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f49bf-fc6d-44ba-a097-dcc0ef0388d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.355304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f49bf-fc6d-44ba-a097-dcc0ef0388d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.355757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f49bf-fc6d-44ba-a097-dcc0ef0388d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.356940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f49bf-fc6d-44ba-a097-dcc0ef0388d3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.369803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ab39d1ca-5c5c-448d-b3dd-bc64606fbc33 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.370993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ab39d1ca-5c5c-448d-b3dd-bc64606fbc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.378839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 782/1567] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.380902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39d1ca-5c5c-448d-b3dd-bc64606fbc33 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.388605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39d1ca-5c5c-448d-b3dd-bc64606fbc33 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.393262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4ea55-462a-45da-adf9-f819fb2542b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.393869 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4ea55-462a-45da-adf9-f819fb2542b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.394141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4ea55-462a-45da-adf9-f819fb2542b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.394404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4ea55-462a-45da-adf9-f819fb2542b2 admin admin] PATH_INFO: `/v3/projects/c0061f213bfd4e3b87a0582b6cec6487/users/2d5e7d5c77084705b1f39406293ab585/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.408915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4ea55-462a-45da-adf9-f819fb2542b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0061f213bfd4e3b87a0582b6cec6487, user_id=2d5e7d5c77084705b1f39406293ab585, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.409948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-93f4ea55-462a-45da-adf9-f819fb2542b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:08:47.409948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Aug 08 18:08:47.409948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: Aug 08 18:08:47.411856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4ea55-462a-45da-adf9-f819fb2542b2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.451477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 786/1568] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:47.452290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f49bf-fc6d-44ba-a097-dcc0ef0388d3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.472817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 783/1569] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:47 2022] PUT /identity/v3/projects/c0061f213bfd4e3b87a0582b6cec6487/users/2d5e7d5c77084705b1f39406293ab585/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:47.476992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4ea55-462a-45da-adf9-f819fb2542b2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.514357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f49bf-fc6d-44ba-a097-dcc0ef0388d3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.534763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4ea55-462a-45da-adf9-f819fb2542b2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.539485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89def8a4-1ff1-4a7b-8e56-5c97f2b0058a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.540065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89def8a4-1ff1-4a7b-8e56-5c97f2b0058a admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.540336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89def8a4-1ff1-4a7b-8e56-5c97f2b0058a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.540632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89def8a4-1ff1-4a7b-8e56-5c97f2b0058a admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.541249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89def8a4-1ff1-4a7b-8e56-5c97f2b0058a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.543176 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89def8a4-1ff1-4a7b-8e56-5c97f2b0058a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.571030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa7a26-6417-4c4a-878b-917f5c6915af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.571581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7a26-6417-4c4a-878b-917f5c6915af admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.571841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7a26-6417-4c4a-878b-917f5c6915af admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.572108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7a26-6417-4c4a-878b-917f5c6915af admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.572816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa7a26-6417-4c4a-878b-917f5c6915af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:08:47.573115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa7a26-6417-4c4a-878b-917f5c6915af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.574755 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa7a26-6417-4c4a-878b-917f5c6915af admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.586129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 787/1570] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:08:47.587265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3778af0-babf-4e21-89ac-96eb7e3159fd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.587547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3778af0-babf-4e21-89ac-96eb7e3159fd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.587818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3778af0-babf-4e21-89ac-96eb7e3159fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.607991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c3778af0-babf-4e21-89ac-96eb7e3159fd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.609255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c3778af0-babf-4e21-89ac-96eb7e3159fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.616850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 788/1571] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.617985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117013ba-1ffc-42f7-b951-e6be9998992c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.618264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117013ba-1ffc-42f7-b951-e6be9998992c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.618534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117013ba-1ffc-42f7-b951-e6be9998992c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.637626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-117013ba-1ffc-42f7-b951-e6be9998992c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.638802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-117013ba-1ffc-42f7-b951-e6be9998992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.649315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 784/1572] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:47.649465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 789/1573] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.649551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89def8a4-1ff1-4a7b-8e56-5c97f2b0058a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.649650 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117013ba-1ffc-42f7-b951-e6be9998992c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.658619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117013ba-1ffc-42f7-b951-e6be9998992c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.695471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:47.711605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89def8a4-1ff1-4a7b-8e56-5c97f2b0058a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.714772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b6c6-9f26-4ae9-96f7-0c31e080454a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.716019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b6c6-9f26-4ae9-96f7-0c31e080454a admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.716406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b6c6-9f26-4ae9-96f7-0c31e080454a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.716735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b6c6-9f26-4ae9-96f7-0c31e080454a admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.717078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1277ee9-e650-41a6-a141-4c32326854bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.717412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b6c6-9f26-4ae9-96f7-0c31e080454a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:08:47.717666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1277ee9-e650-41a6-a141-4c32326854bf admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.717947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1277ee9-e650-41a6-a141-4c32326854bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.718063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b6c6-9f26-4ae9-96f7-0c31e080454a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.718219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1277ee9-e650-41a6-a141-4c32326854bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.718826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1277ee9-e650-41a6-a141-4c32326854bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.719856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b6c6-9f26-4ae9-96f7-0c31e080454a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.720749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1277ee9-e650-41a6-a141-4c32326854bf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.726067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:47.731710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 790/1574] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:08:47.732355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b6c6-9f26-4ae9-96f7-0c31e080454a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.743612 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b6c6-9f26-4ae9-96f7-0c31e080454a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.748976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d093ce-a54a-418d-85fc-29c4453b8048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.749558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d093ce-a54a-418d-85fc-29c4453b8048 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.749834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d093ce-a54a-418d-85fc-29c4453b8048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.750100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d093ce-a54a-418d-85fc-29c4453b8048 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.750708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d093ce-a54a-418d-85fc-29c4453b8048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.752712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d093ce-a54a-418d-85fc-29c4453b8048 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.783688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 785/1575] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:47.784945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eed2e4-1813-43f9-8700-abaa6603c2ce None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.785224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eed2e4-1813-43f9-8700-abaa6603c2ce None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.785595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eed2e4-1813-43f9-8700-abaa6603c2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.808163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-70eed2e4-1813-43f9-8700-abaa6603c2ce None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:47.809449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-70eed2e4-1813-43f9-8700-abaa6603c2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:47.833453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 791/1576] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:47.834222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d093ce-a54a-418d-85fc-29c4453b8048 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.844919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d093ce-a54a-418d-85fc-29c4453b8048 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.868982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 786/1577] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:47.869824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eed2e4-1813-43f9-8700-abaa6603c2ce None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.902844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4528e-3795-4e19-a5f0-b8ae179da334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.903452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4528e-3795-4e19-a5f0-b8ae179da334 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.903729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4528e-3795-4e19-a5f0-b8ae179da334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.903993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4528e-3795-4e19-a5f0-b8ae179da334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.904633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4528e-3795-4e19-a5f0-b8ae179da334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.906534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4528e-3795-4e19-a5f0-b8ae179da334 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.923662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eed2e4-1813-43f9-8700-abaa6603c2ce None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.927327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d220a8-2977-400e-8725-17e2e73ea782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.927918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d220a8-2977-400e-8725-17e2e73ea782 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.928187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d220a8-2977-400e-8725-17e2e73ea782 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.928537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d220a8-2977-400e-8725-17e2e73ea782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.938244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d220a8-2977-400e-8725-17e2e73ea782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.939662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d220a8-2977-400e-8725-17e2e73ea782 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.948981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 787/1578] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:47.950100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d220a8-2977-400e-8725-17e2e73ea782 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:47.959857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d220a8-2977-400e-8725-17e2e73ea782 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:47.964450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce75475-ec3a-4789-8bc4-aaabea2a0baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:47.965584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce75475-ec3a-4789-8bc4-aaabea2a0baa admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.965860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce75475-ec3a-4789-8bc4-aaabea2a0baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.966125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce75475-ec3a-4789-8bc4-aaabea2a0baa admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:47.966727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce75475-ec3a-4789-8bc4-aaabea2a0baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:47.968705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce75475-ec3a-4789-8bc4-aaabea2a0baa admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:47.984839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 792/1579] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/projects => generated 391 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:47.986266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5155182-f90b-435e-9a6d-8280f2c88dbc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:47.986539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5155182-f90b-435e-9a6d-8280f2c88dbc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:47.986799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5155182-f90b-435e-9a6d-8280f2c88dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.005595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d5155182-f90b-435e-9a6d-8280f2c88dbc None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:48.006772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d5155182-f90b-435e-9a6d-8280f2c88dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:48.063811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 793/1580] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:48.064657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5155182-f90b-435e-9a6d-8280f2c88dbc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.073523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5155182-f90b-435e-9a6d-8280f2c88dbc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.078563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5468b-0641-443c-be90-15d76925c91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.079139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5468b-0641-443c-be90-15d76925c91c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.079408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5468b-0641-443c-be90-15d76925c91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.079670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5468b-0641-443c-be90-15d76925c91c admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.080300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5468b-0641-443c-be90-15d76925c91c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.082484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5468b-0641-443c-be90-15d76925c91c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.110880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 788/1581] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:08:47 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:48.110880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce75475-ec3a-4789-8bc4-aaabea2a0baa admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.119047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce75475-ec3a-4789-8bc4-aaabea2a0baa admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.123841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98fb23-f07f-45c1-a2a0-a1e50108f445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.124425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98fb23-f07f-45c1-a2a0-a1e50108f445 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.124732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98fb23-f07f-45c1-a2a0-a1e50108f445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.125002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98fb23-f07f-45c1-a2a0-a1e50108f445 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.125619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98fb23-f07f-45c1-a2a0-a1e50108f445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.127541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98fb23-f07f-45c1-a2a0-a1e50108f445 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.172130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 794/1582] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:08:48 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:48.173019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5468b-0641-443c-be90-15d76925c91c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.193573 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:48.229931 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 789/1583] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:08:48 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:48.231590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98fb23-f07f-45c1-a2a0-a1e50108f445 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.237499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5468b-0641-443c-be90-15d76925c91c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.242217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b1c18-a56c-4b5b-8556-628f835d1144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.242828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b1c18-a56c-4b5b-8556-628f835d1144 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.243104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b1c18-a56c-4b5b-8556-628f835d1144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.243376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b1c18-a56c-4b5b-8556-628f835d1144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.244010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b1c18-a56c-4b5b-8556-628f835d1144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.245641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b1c18-a56c-4b5b-8556-628f835d1144 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.255432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 795/1584] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:48.256207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b1c18-a56c-4b5b-8556-628f835d1144 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.267498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b1c18-a56c-4b5b-8556-628f835d1144 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.272109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8daf03-2d6e-4151-a388-27b33cd36084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.272726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8daf03-2d6e-4151-a388-27b33cd36084 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.273000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8daf03-2d6e-4151-a388-27b33cd36084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.273264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8daf03-2d6e-4151-a388-27b33cd36084 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.273879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8daf03-2d6e-4151-a388-27b33cd36084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.275791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8daf03-2d6e-4151-a388-27b33cd36084 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.297567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98fb23-f07f-45c1-a2a0-a1e50108f445 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.305221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:48.347837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9852fd-1c73-4c64-8f2b-94ba53bfa5ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.348403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9852fd-1c73-4c64-8f2b-94ba53bfa5ed service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.348749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9852fd-1c73-4c64-8f2b-94ba53bfa5ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.349014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9852fd-1c73-4c64-8f2b-94ba53bfa5ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.364067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9852fd-1c73-4c64-8f2b-94ba53bfa5ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.365590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9852fd-1c73-4c64-8f2b-94ba53bfa5ed service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.375702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 790/1585] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:48.376725 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9852fd-1c73-4c64-8f2b-94ba53bfa5ed service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.386433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9852fd-1c73-4c64-8f2b-94ba53bfa5ed service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.391436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 796/1586] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:08:48 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:48.391941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e723b18-7db6-4cc8-ba9c-a05f9cc5d8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.392537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8daf03-2d6e-4151-a388-27b33cd36084 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.392679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e723b18-7db6-4cc8-ba9c-a05f9cc5d8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.392679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e723b18-7db6-4cc8-ba9c-a05f9cc5d8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.392859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e723b18-7db6-4cc8-ba9c-a05f9cc5d8f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.393456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e723b18-7db6-4cc8-ba9c-a05f9cc5d8f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:08:48.393755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e723b18-7db6-4cc8-ba9c-a05f9cc5d8f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.395345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e723b18-7db6-4cc8-ba9c-a05f9cc5d8f0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.403534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8daf03-2d6e-4151-a388-27b33cd36084 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.409179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 791/1587] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:08:48.410316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e723b18-7db6-4cc8-ba9c-a05f9cc5d8f0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.416783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a5dc9-a2ee-470d-8dc7-56f28065438d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.419382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a5dc9-a2ee-470d-8dc7-56f28065438d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.419382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a5dc9-a2ee-470d-8dc7-56f28065438d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.419382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a5dc9-a2ee-470d-8dc7-56f28065438d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.419382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a5dc9-a2ee-470d-8dc7-56f28065438d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.420938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a5dc9-a2ee-470d-8dc7-56f28065438d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.421070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e723b18-7db6-4cc8-ba9c-a05f9cc5d8f0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.425675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b738951-375f-44ff-98fa-e6de7bfe2ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.426179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b738951-375f-44ff-98fa-e6de7bfe2ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.426475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b738951-375f-44ff-98fa-e6de7bfe2ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.426738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b738951-375f-44ff-98fa-e6de7bfe2ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.427373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b738951-375f-44ff-98fa-e6de7bfe2ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.429418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b738951-375f-44ff-98fa-e6de7bfe2ec5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.432025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 797/1588] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:48.432874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a5dc9-a2ee-470d-8dc7-56f28065438d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.443859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a5dc9-a2ee-470d-8dc7-56f28065438d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.448605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136c355-1ec8-47a9-b92c-7de30afa62ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.449181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136c355-1ec8-47a9-b92c-7de30afa62ce admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.450216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136c355-1ec8-47a9-b92c-7de30afa62ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.450482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136c355-1ec8-47a9-b92c-7de30afa62ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.451242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136c355-1ec8-47a9-b92c-7de30afa62ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.452879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136c355-1ec8-47a9-b92c-7de30afa62ce admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.462671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 798/1589] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:48.463478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136c355-1ec8-47a9-b92c-7de30afa62ce admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.473626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136c355-1ec8-47a9-b92c-7de30afa62ce admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.478866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14fce4-396c-4cb0-a17a-b541aac97d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.479616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14fce4-396c-4cb0-a17a-b541aac97d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.480065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14fce4-396c-4cb0-a17a-b541aac97d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.480964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14fce4-396c-4cb0-a17a-b541aac97d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.480964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14fce4-396c-4cb0-a17a-b541aac97d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.482560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14fce4-396c-4cb0-a17a-b541aac97d79 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.488128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 792/1590] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:08:48 2022] POST /identity/v3/users => generated 295 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:48.490052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b738951-375f-44ff-98fa-e6de7bfe2ec5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.494732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 799/1591] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:48.496055 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14fce4-396c-4cb0-a17a-b541aac97d79 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.501751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b738951-375f-44ff-98fa-e6de7bfe2ec5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.506493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59450a2-b6ce-4f4f-b198-96deae424371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.507099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59450a2-b6ce-4f4f-b198-96deae424371 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.507386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59450a2-b6ce-4f4f-b198-96deae424371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.507623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59450a2-b6ce-4f4f-b198-96deae424371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.507887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14fce4-396c-4cb0-a17a-b541aac97d79 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.508264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59450a2-b6ce-4f4f-b198-96deae424371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.509892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59450a2-b6ce-4f4f-b198-96deae424371 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.513748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969a8e5-f165-4e17-93f0-71ce4c0cfe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.514360 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969a8e5-f165-4e17-93f0-71ce4c0cfe39 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.514694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969a8e5-f165-4e17-93f0-71ce4c0cfe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.514972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969a8e5-f165-4e17-93f0-71ce4c0cfe39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.515757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969a8e5-f165-4e17-93f0-71ce4c0cfe39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.517806 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969a8e5-f165-4e17-93f0-71ce4c0cfe39 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.521518 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 793/1592] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:48.522409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59450a2-b6ce-4f4f-b198-96deae424371 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.534979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59450a2-b6ce-4f4f-b198-96deae424371 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.539926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed5dca-f947-4699-8a05-6a3c1d22cb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.540586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed5dca-f947-4699-8a05-6a3c1d22cb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.543919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed5dca-f947-4699-8a05-6a3c1d22cb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.543919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed5dca-f947-4699-8a05-6a3c1d22cb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.543919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed5dca-f947-4699-8a05-6a3c1d22cb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.543919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed5dca-f947-4699-8a05-6a3c1d22cb07 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.553967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 794/1593] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:48.553967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed5dca-f947-4699-8a05-6a3c1d22cb07 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.564329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed5dca-f947-4699-8a05-6a3c1d22cb07 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.569934 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907e694-000a-435c-9b57-7139a9c5fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.570576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907e694-000a-435c-9b57-7139a9c5fe49 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.570849 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907e694-000a-435c-9b57-7139a9c5fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.571114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907e694-000a-435c-9b57-7139a9c5fe49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.571737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907e694-000a-435c-9b57-7139a9c5fe49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.573325 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907e694-000a-435c-9b57-7139a9c5fe49 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.583795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 795/1594] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:48.587700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907e694-000a-435c-9b57-7139a9c5fe49 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.595832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 800/1595] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:08:48 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:48.598983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907e694-000a-435c-9b57-7139a9c5fe49 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.600059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969a8e5-f165-4e17-93f0-71ce4c0cfe39 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.686698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950a78c-7b89-4688-ac66-8ca502c749e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.687400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950a78c-7b89-4688-ac66-8ca502c749e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.687747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950a78c-7b89-4688-ac66-8ca502c749e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.688083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950a78c-7b89-4688-ac66-8ca502c749e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.688892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950a78c-7b89-4688-ac66-8ca502c749e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.691179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950a78c-7b89-4688-ac66-8ca502c749e7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.702327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969a8e5-f165-4e17-93f0-71ce4c0cfe39 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.704819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 796/1596] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:48.706174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950a78c-7b89-4688-ac66-8ca502c749e7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.711883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7591c4-4052-497b-bd87-3ff67cdf1aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.713505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7591c4-4052-497b-bd87-3ff67cdf1aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.714546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7591c4-4052-497b-bd87-3ff67cdf1aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.715700 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7591c4-4052-497b-bd87-3ff67cdf1aee admin admin] PATH_INFO: `/v3/projects/5de58eb192de494b809a04e638e2a291/users/caf8d88ca4da4515bd9f1bec64da3375/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.717481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950a78c-7b89-4688-ac66-8ca502c749e7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.723541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021501e3-cdbd-47d2-9582-9cd8ca9b6bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.724262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021501e3-cdbd-47d2-9582-9cd8ca9b6bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.724558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021501e3-cdbd-47d2-9582-9cd8ca9b6bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.724867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021501e3-cdbd-47d2-9582-9cd8ca9b6bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.725542 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021501e3-cdbd-47d2-9582-9cd8ca9b6bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.727251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021501e3-cdbd-47d2-9582-9cd8ca9b6bdd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.740564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 797/1597] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:48.740564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021501e3-cdbd-47d2-9582-9cd8ca9b6bdd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.743927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7591c4-4052-497b-bd87-3ff67cdf1aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de58eb192de494b809a04e638e2a291, user_id=caf8d88ca4da4515bd9f1bec64da3375, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.747487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7591c4-4052-497b-bd87-3ff67cdf1aee admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.749899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021501e3-cdbd-47d2-9582-9cd8ca9b6bdd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.754644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0da6e8-26f2-43e7-8bab-1cc582d48f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.760796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0da6e8-26f2-43e7-8bab-1cc582d48f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.760796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0da6e8-26f2-43e7-8bab-1cc582d48f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.760796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0da6e8-26f2-43e7-8bab-1cc582d48f86 admin admin] PATH_INFO: `/v3/projects/3b463a1983da46d398345bee415f1a55/users/c9cee55af1bb4dd7a7f7956ef088d59f/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.772291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0da6e8-26f2-43e7-8bab-1cc582d48f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b463a1983da46d398345bee415f1a55, user_id=c9cee55af1bb4dd7a7f7956ef088d59f, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.775099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0da6e8-26f2-43e7-8bab-1cc582d48f86 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.830507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 801/1598] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:48 2022] PUT /identity/v3/projects/5de58eb192de494b809a04e638e2a291/users/caf8d88ca4da4515bd9f1bec64da3375/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:48.830507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7591c4-4052-497b-bd87-3ff67cdf1aee admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.857241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 798/1599] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:48 2022] PUT /identity/v3/projects/3b463a1983da46d398345bee415f1a55/users/c9cee55af1bb4dd7a7f7956ef088d59f/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:48.858013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0da6e8-26f2-43e7-8bab-1cc582d48f86 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:48.869428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:48.875473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:48.907319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7591c4-4052-497b-bd87-3ff67cdf1aee admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.931780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0da6e8-26f2-43e7-8bab-1cc582d48f86 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:48.938123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db5bad-7cbd-4f3d-b1be-e390216c5402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.938758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db5bad-7cbd-4f3d-b1be-e390216c5402 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.939032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db5bad-7cbd-4f3d-b1be-e390216c5402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.939502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db5bad-7cbd-4f3d-b1be-e390216c5402 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.940115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db5bad-7cbd-4f3d-b1be-e390216c5402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.942097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db5bad-7cbd-4f3d-b1be-e390216c5402 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:48.965600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190994c-820f-454a-9dc5-710565d33523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:48.966253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190994c-820f-454a-9dc5-710565d33523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:48.966531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190994c-820f-454a-9dc5-710565d33523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:48.966799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190994c-820f-454a-9dc5-710565d33523 admin admin] PATH_INFO: `/v3/projects/16af2915d7254078ad58c7592a20f126/users/f880bd118d3b4048954f975d3d907bf8/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:48.982147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190994c-820f-454a-9dc5-710565d33523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16af2915d7254078ad58c7592a20f126, user_id=f880bd118d3b4048954f975d3d907bf8, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:48.984919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190994c-820f-454a-9dc5-710565d33523 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:49.054031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 799/1600] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:08:48 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:49.054031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db5bad-7cbd-4f3d-b1be-e390216c5402 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:49.072807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db5bad-7cbd-4f3d-b1be-e390216c5402 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:49.079065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340e63e-753f-401a-85af-65eb339bebe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:49.079707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340e63e-753f-401a-85af-65eb339bebe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.080016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340e63e-753f-401a-85af-65eb339bebe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.080384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340e63e-753f-401a-85af-65eb339bebe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.081056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340e63e-753f-401a-85af-65eb339bebe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:49.082799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340e63e-753f-401a-85af-65eb339bebe4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:49.093180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 800/1601] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:49.094040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340e63e-753f-401a-85af-65eb339bebe4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:49.105220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 802/1602] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:48 2022] PUT /identity/v3/projects/16af2915d7254078ad58c7592a20f126/users/f880bd118d3b4048954f975d3d907bf8/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:49.107475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc52e1-42a8-445e-bb62-4d511481daa6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.107830 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc52e1-42a8-445e-bb62-4d511481daa6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.108219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc52e1-42a8-445e-bb62-4d511481daa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.122373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340e63e-753f-401a-85af-65eb339bebe4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:49.134933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c8fc52e1-42a8-445e-bb62-4d511481daa6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:49.136230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c8fc52e1-42a8-445e-bb62-4d511481daa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:49.182599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fcdb3-d9a1-472b-8219-f5fbe751cdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:49.183251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fcdb3-d9a1-472b-8219-f5fbe751cdfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.183519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fcdb3-d9a1-472b-8219-f5fbe751cdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.183843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fcdb3-d9a1-472b-8219-f5fbe751cdfc admin admin] PATH_INFO: `/v3/projects/34b812a3f98747eeac35fbc27a342ab0/users/4c7a6c1d08554db09d65394ec338f8b5/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.198578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fcdb3-d9a1-472b-8219-f5fbe751cdfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b812a3f98747eeac35fbc27a342ab0, user_id=4c7a6c1d08554db09d65394ec338f8b5, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:49.200803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 803/1603] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:49.201930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f85d6-ef28-4634-ae3a-e4ca41064f4e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.202213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f85d6-ef28-4634-ae3a-e4ca41064f4e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.202510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f85d6-ef28-4634-ae3a-e4ca41064f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.205381 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fcdb3-d9a1-472b-8219-f5fbe751cdfc admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:49.232266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a57f85d6-ef28-4634-ae3a-e4ca41064f4e None None] MFA Rules not processed for user `c9cee55af1bb4dd7a7f7956ef088d59f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:49.233586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a57f85d6-ef28-4634-ae3a-e4ca41064f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:49.280079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 801/1604] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:49 2022] PUT /identity/v3/projects/34b812a3f98747eeac35fbc27a342ab0/users/4c7a6c1d08554db09d65394ec338f8b5/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:49.281110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fcdb3-d9a1-472b-8219-f5fbe751cdfc admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:49.288759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:49.317801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 804/1605] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:49.318797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f85d6-ef28-4634-ae3a-e4ca41064f4e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:49.331019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f85d6-ef28-4634-ae3a-e4ca41064f4e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:49.393792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fcdb3-d9a1-472b-8219-f5fbe751cdfc admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:49.408933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ec19-a860-457e-b325-168b668c2a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:49.409493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ec19-a860-457e-b325-168b668c2a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.409769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ec19-a860-457e-b325-168b668c2a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.410035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ec19-a860-457e-b325-168b668c2a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.413116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847ec19-a860-457e-b325-168b668c2a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:49.415981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847ec19-a860-457e-b325-168b668c2a23 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:49.430824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 802/1606] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:49.432086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883645c-30a9-4e13-8013-ee677509b0f2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.432373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883645c-30a9-4e13-8013-ee677509b0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.432669 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883645c-30a9-4e13-8013-ee677509b0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.448720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed635e9-f31d-475c-97a5-6a60ce335707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:49.451151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed635e9-f31d-475c-97a5-6a60ce335707 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.451151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed635e9-f31d-475c-97a5-6a60ce335707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.451847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed635e9-f31d-475c-97a5-6a60ce335707 admin admin] PATH_INFO: `/v3/projects/cf197791c8734703b20784498cd8dcd5/users/2ec53f935a6c41f2af67e9fd2e13e12c/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.453940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:49.464669 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f883645c-30a9-4e13-8013-ee677509b0f2 None None] MFA Rules not processed for user `f880bd118d3b4048954f975d3d907bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:49.468462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f883645c-30a9-4e13-8013-ee677509b0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:49.469821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed635e9-f31d-475c-97a5-6a60ce335707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf197791c8734703b20784498cd8dcd5, user_id=2ec53f935a6c41f2af67e9fd2e13e12c, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:49.472601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed635e9-f31d-475c-97a5-6a60ce335707 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:49.525218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 805/1607] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:49 2022] PUT /identity/v3/projects/cf197791c8734703b20784498cd8dcd5/users/2ec53f935a6c41f2af67e9fd2e13e12c/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:49.526490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed635e9-f31d-475c-97a5-6a60ce335707 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:49.532001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8ed635e9-f31d-475c-97a5-6a60ce335707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:49.586511 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 803/1608] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:49.587690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f0e0d-984c-4120-8eca-aa79d7d75773 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.588040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f0e0d-984c-4120-8eca-aa79d7d75773 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.588318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f0e0d-984c-4120-8eca-aa79d7d75773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.602109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed635e9-f31d-475c-97a5-6a60ce335707 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:49.609421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24ad4a-cccc-4534-8c48-4da2cadeac62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:49.609421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ad4a-cccc-4534-8c48-4da2cadeac62 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.609421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ad4a-cccc-4534-8c48-4da2cadeac62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.609421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ad4a-cccc-4534-8c48-4da2cadeac62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.611217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3e5f0e0d-984c-4120-8eca-aa79d7d75773 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:49.612657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3e5f0e0d-984c-4120-8eca-aa79d7d75773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:49.682631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 804/1609] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:49.688291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4087d6-d10c-489a-aa77-94b7757fa45f None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.688291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4087d6-d10c-489a-aa77-94b7757fa45f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.688291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4087d6-d10c-489a-aa77-94b7757fa45f None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.688291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 805/1610] 10.222.0.50 () {54 vars in 962 bytes} [Mon Aug 8 18:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:08:49.688291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4087d6-d10c-489a-aa77-94b7757fa45f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:49.701959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4087d6-d10c-489a-aa77-94b7757fa45f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:49.708408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee59e-001f-4aa8-9873-d7bc93c105e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:49.709343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee59e-001f-4aa8-9873-d7bc93c105e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.709714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee59e-001f-4aa8-9873-d7bc93c105e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.710092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee59e-001f-4aa8-9873-d7bc93c105e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.710703 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24ad4a-cccc-4534-8c48-4da2cadeac62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:49.710931 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee59e-001f-4aa8-9873-d7bc93c105e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:49.712283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24ad4a-cccc-4534-8c48-4da2cadeac62 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:49.712995 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee59e-001f-4aa8-9873-d7bc93c105e4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:49.723580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 806/1611] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:49.724639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 806/1612] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:49.725982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead88b3d-9dee-415a-a80f-c4d5cec4e1ea None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.726263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead88b3d-9dee-415a-a80f-c4d5cec4e1ea None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.726531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee59e-001f-4aa8-9873-d7bc93c105e4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:49.726726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead88b3d-9dee-415a-a80f-c4d5cec4e1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.738402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee59e-001f-4aa8-9873-d7bc93c105e4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:49.743793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d12c9-af1b-4950-ae28-040cb6b4310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:49.744042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d12c9-af1b-4950-ae28-040cb6b4310a admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.744318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d12c9-af1b-4950-ae28-040cb6b4310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.744624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d12c9-af1b-4950-ae28-040cb6b4310a admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.745233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d12c9-af1b-4950-ae28-040cb6b4310a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:49.747351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d12c9-af1b-4950-ae28-040cb6b4310a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:49.753685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ead88b3d-9dee-415a-a80f-c4d5cec4e1ea None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:49.755377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ead88b3d-9dee-415a-a80f-c4d5cec4e1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:49.768321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 807/1613] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:49.770339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c44ed1-f2f0-4300-a432-f3204ecb4a99 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:49.770767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c44ed1-f2f0-4300-a432-f3204ecb4a99 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:49.771324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c44ed1-f2f0-4300-a432-f3204ecb4a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:49.812868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:49.827960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-03c44ed1-f2f0-4300-a432-f3204ecb4a99 None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:49.829246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-03c44ed1-f2f0-4300-a432-f3204ecb4a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:49.909145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 807/1614] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:08:49 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:49.913100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d12c9-af1b-4950-ae28-040cb6b4310a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:49.927707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 808/1615] 10.222.0.50 () {58 vars in 1059 bytes} [Mon Aug 8 18:08:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:49.929013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c44ed1-f2f0-4300-a432-f3204ecb4a99 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:49.941517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c44ed1-f2f0-4300-a432-f3204ecb4a99 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:50.001335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d12c9-af1b-4950-ae28-040cb6b4310a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:50.008965 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7b68f-2916-4a03-a461-bedd75de752a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:50.009597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b68f-2916-4a03-a461-bedd75de752a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.009880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b68f-2916-4a03-a461-bedd75de752a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.010158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b68f-2916-4a03-a461-bedd75de752a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.039382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1b1f4-e505-4b0a-a27e-859b2c241fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:50.041254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1b1f4-e505-4b0a-a27e-859b2c241fff admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.041254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1b1f4-e505-4b0a-a27e-859b2c241fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.041500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1b1f4-e505-4b0a-a27e-859b2c241fff admin admin] PATH_INFO: `/v3/projects/e023a8ae7a5340f2b652682525ac8a30/users/491f22a4783f48d298a2507a0b10ddbd/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.067670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1b1f4-e505-4b0a-a27e-859b2c241fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e023a8ae7a5340f2b652682525ac8a30, user_id=491f22a4783f48d298a2507a0b10ddbd, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:50.070448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1b1f4-e505-4b0a-a27e-859b2c241fff admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:50.099226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7b68f-2916-4a03-a461-bedd75de752a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:50.101223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7b68f-2916-4a03-a461-bedd75de752a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:50.113252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 808/1616] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:50.114069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7b68f-2916-4a03-a461-bedd75de752a service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:50.124402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7b68f-2916-4a03-a461-bedd75de752a service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:50.130770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef0303-6913-48b3-9689-ec5802315c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:50.131512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0303-6913-48b3-9689-ec5802315c38 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.132535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0303-6913-48b3-9689-ec5802315c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.134594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0303-6913-48b3-9689-ec5802315c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.155502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef0303-6913-48b3-9689-ec5802315c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:50.158203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef0303-6913-48b3-9689-ec5802315c38 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:50.177654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:50.185100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 809/1617] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:49 2022] PUT /identity/v3/projects/e023a8ae7a5340f2b652682525ac8a30/users/491f22a4783f48d298a2507a0b10ddbd/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:50.186112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1b1f4-e505-4b0a-a27e-859b2c241fff admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:50.201211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 809/1618] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:50.202401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef0303-6913-48b3-9689-ec5802315c38 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:50.310829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef0303-6913-48b3-9689-ec5802315c38 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:50.337146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1b1f4-e505-4b0a-a27e-859b2c241fff admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:50.352472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25de42-a374-4b5f-9894-238368c4fb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:50.354463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25de42-a374-4b5f-9894-238368c4fb88 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.354743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25de42-a374-4b5f-9894-238368c4fb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.355015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25de42-a374-4b5f-9894-238368c4fb88 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.358722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25de42-a374-4b5f-9894-238368c4fb88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:50.364292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25de42-a374-4b5f-9894-238368c4fb88 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:50.436759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1078293-ae6f-4421-a9bf-cb4896edb956 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:50.437346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1078293-ae6f-4421-a9bf-cb4896edb956 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.437558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1078293-ae6f-4421-a9bf-cb4896edb956 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.437832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1078293-ae6f-4421-a9bf-cb4896edb956 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.508624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 810/1619] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:08:50 2022] POST /identity/v3/users => generated 293 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:08:50.509996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2a81d-0180-4fb5-82bd-8347b223558f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.510470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2a81d-0180-4fb5-82bd-8347b223558f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.510836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2a81d-0180-4fb5-82bd-8347b223558f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.517275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1078293-ae6f-4421-a9bf-cb4896edb956 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:50.519858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1078293-ae6f-4421-a9bf-cb4896edb956 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:50.544630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 810/1620] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:50.546890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1078293-ae6f-4421-a9bf-cb4896edb956 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:50.572568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1078293-ae6f-4421-a9bf-cb4896edb956 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:50.583634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-57d2a81d-0180-4fb5-82bd-8347b223558f None None] MFA Rules not processed for user `491f22a4783f48d298a2507a0b10ddbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:50.586532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440adcd1-f099-4f7d-83aa-bd31ce43cdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:50.586862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-57d2a81d-0180-4fb5-82bd-8347b223558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:50.587373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440adcd1-f099-4f7d-83aa-bd31ce43cdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.587657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440adcd1-f099-4f7d-83aa-bd31ce43cdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.587925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440adcd1-f099-4f7d-83aa-bd31ce43cdc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.588596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440adcd1-f099-4f7d-83aa-bd31ce43cdc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:50.590984 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440adcd1-f099-4f7d-83aa-bd31ce43cdc2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:50.615876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 811/1621] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:50.620533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440adcd1-f099-4f7d-83aa-bd31ce43cdc2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:50.644323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440adcd1-f099-4f7d-83aa-bd31ce43cdc2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:50.664413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9574135-abf1-4fcd-978e-9f470e9580d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:50.671947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9574135-abf1-4fcd-978e-9f470e9580d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.672390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9574135-abf1-4fcd-978e-9f470e9580d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.672693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9574135-abf1-4fcd-978e-9f470e9580d2 admin admin] PATH_INFO: `/v3/projects/fa30ab0e6e924bed808ce918e654e8e2/users/6255db3e42844d6388f6018ef6b3c03f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.713392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9574135-abf1-4fcd-978e-9f470e9580d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa30ab0e6e924bed808ce918e654e8e2, user_id=6255db3e42844d6388f6018ef6b3c03f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:50.722427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9574135-abf1-4fcd-978e-9f470e9580d2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:50.773961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 811/1622] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:50.782376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:50.805504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96cff-538d-44f6-bf97-12f8c8f9ec83 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.805950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96cff-538d-44f6-bf97-12f8c8f9ec83 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.806063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96cff-538d-44f6-bf97-12f8c8f9ec83 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.807972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 812/1623] 10.222.0.50 () {54 vars in 962 bytes} [Mon Aug 8 18:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:08:50.820370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835886df-97e6-464e-b60c-d274e0aaa5ac None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.820731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835886df-97e6-464e-b60c-d274e0aaa5ac None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.821313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835886df-97e6-464e-b60c-d274e0aaa5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.857763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 812/1624] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:50 2022] PUT /identity/v3/projects/fa30ab0e6e924bed808ce918e654e8e2/users/6255db3e42844d6388f6018ef6b3c03f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:50.861600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9574135-abf1-4fcd-978e-9f470e9580d2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:50.869165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-835886df-97e6-464e-b60c-d274e0aaa5ac None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:50.870493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-835886df-97e6-464e-b60c-d274e0aaa5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:50.978749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9574135-abf1-4fcd-978e-9f470e9580d2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:50.991220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5b751-47e2-4c11-8952-121abf2368c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:50.992729 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5b751-47e2-4c11-8952-121abf2368c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:50.993007 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5b751-47e2-4c11-8952-121abf2368c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:50.993273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5b751-47e2-4c11-8952-121abf2368c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:50.996952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5b751-47e2-4c11-8952-121abf2368c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:50.997427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 813/1625] 10.222.0.50 () {58 vars in 1059 bytes} [Mon Aug 8 18:08:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:08:50.998941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5b751-47e2-4c11-8952-121abf2368c7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:51.002005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835886df-97e6-464e-b60c-d274e0aaa5ac None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:51.016587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835886df-97e6-464e-b60c-d274e0aaa5ac None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:51.024089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd83ac5-bcac-4d18-b339-c4eb0e43e4f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:51.030342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 813/1626] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:08:51.030487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd83ac5-bcac-4d18-b339-c4eb0e43e4f1 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.030487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd83ac5-bcac-4d18-b339-c4eb0e43e4f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.030487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd83ac5-bcac-4d18-b339-c4eb0e43e4f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:51.030724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5b751-47e2-4c11-8952-121abf2368c7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:51.129776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5b751-47e2-4c11-8952-121abf2368c7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:51.129776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284eb40-ebf6-4dfa-9812-5df6a0a34483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:51.129776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284eb40-ebf6-4dfa-9812-5df6a0a34483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.129776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284eb40-ebf6-4dfa-9812-5df6a0a34483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.130979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284eb40-ebf6-4dfa-9812-5df6a0a34483 admin admin] PATH_INFO: `/v3/projects/fa30ab0e6e924bed808ce918e654e8e2/users/6255db3e42844d6388f6018ef6b3c03f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:51.130979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284eb40-ebf6-4dfa-9812-5df6a0a34483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa30ab0e6e924bed808ce918e654e8e2, user_id=6255db3e42844d6388f6018ef6b3c03f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:51.130979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284eb40-ebf6-4dfa-9812-5df6a0a34483 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:51.147221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 814/1627] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:08:51 2022] PUT /identity/v3/projects/fa30ab0e6e924bed808ce918e654e8e2/users/6255db3e42844d6388f6018ef6b3c03f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:08:51.157330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e476ef-e50d-47cb-b686-ce99ee78efb1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.157620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e476ef-e50d-47cb-b686-ce99ee78efb1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.157875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e476ef-e50d-47cb-b686-ce99ee78efb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:51.215681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d2e476ef-e50d-47cb-b686-ce99ee78efb1 None None] MFA Rules not processed for user `6255db3e42844d6388f6018ef6b3c03f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:51.216917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d2e476ef-e50d-47cb-b686-ce99ee78efb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:51.221724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:51.250400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd83ac5-bcac-4d18-b339-c4eb0e43e4f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:51.252259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd83ac5-bcac-4d18-b339-c4eb0e43e4f1 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:51.292973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 814/1628] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:51.294895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b088a-1ef2-4a1d-a4e2-6106953518d0 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.295177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b088a-1ef2-4a1d-a4e2-6106953518d0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.295450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b088a-1ef2-4a1d-a4e2-6106953518d0 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:51.297027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 815/1629] 10.222.0.50 () {54 vars in 912 bytes} [Mon Aug 8 18:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:08:51.302092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab068e4-8412-4a3f-8ce9-7626e51c360e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.302371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab068e4-8412-4a3f-8ce9-7626e51c360e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.302640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab068e4-8412-4a3f-8ce9-7626e51c360e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:51.338629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1ab068e4-8412-4a3f-8ce9-7626e51c360e None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:51.340388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1ab068e4-8412-4a3f-8ce9-7626e51c360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:51.396752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 815/1630] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:08:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:08:51.408252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e476ef-e50d-47cb-b686-ce99ee78efb1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:51.501026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 816/1631] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:08:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:51.516748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab068e4-8412-4a3f-8ce9-7626e51c360e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:51.533174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab068e4-8412-4a3f-8ce9-7626e51c360e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:51.535394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e476ef-e50d-47cb-b686-ce99ee78efb1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:51.541584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461fa77-6393-4b5d-ac22-28aa96545fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:51.542858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461fa77-6393-4b5d-ac22-28aa96545fab service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.543988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461fa77-6393-4b5d-ac22-28aa96545fab service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.545627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461fa77-6393-4b5d-ac22-28aa96545fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:51.545910 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d5df1-1209-4525-9123-c1bf4abdcecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:51.547307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d5df1-1209-4525-9123-c1bf4abdcecb service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.548089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d5df1-1209-4525-9123-c1bf4abdcecb service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.549557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d5df1-1209-4525-9123-c1bf4abdcecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:51.575941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461fa77-6393-4b5d-ac22-28aa96545fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:51.576085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d5df1-1209-4525-9123-c1bf4abdcecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:51.577559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461fa77-6393-4b5d-ac22-28aa96545fab service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:51.577815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d5df1-1209-4525-9123-c1bf4abdcecb service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:51.600469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 816/1632] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:51.603335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 817/1633] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:51.682984 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada4a28-d20d-4fa6-be06-efd7d882566b None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.685076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada4a28-d20d-4fa6-be06-efd7d882566b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.685076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada4a28-d20d-4fa6-be06-efd7d882566b None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:51.687214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 817/1634] 10.222.0.50 () {54 vars in 912 bytes} [Mon Aug 8 18:08:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:08:51.698380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105846c-148e-4aaf-8a8a-de35ab2b1d0e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.698667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105846c-148e-4aaf-8a8a-de35ab2b1d0e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.698958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105846c-148e-4aaf-8a8a-de35ab2b1d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:51.759395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9105846c-148e-4aaf-8a8a-de35ab2b1d0e None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:08:51.766635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9105846c-148e-4aaf-8a8a-de35ab2b1d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:08:51.809452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 818/1635] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:08:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:08:51.858595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:51.943782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada4a28-d20d-4fa6-be06-efd7d882566b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:51.947843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:08:51.967784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada4a28-d20d-4fa6-be06-efd7d882566b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:51.977813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e8092-55f3-4a9c-8d99-3506e1742eff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:51.978914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e8092-55f3-4a9c-8d99-3506e1742eff service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:51.979197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e8092-55f3-4a9c-8d99-3506e1742eff service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:51.979466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e8092-55f3-4a9c-8d99-3506e1742eff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:52.010178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e8092-55f3-4a9c-8d99-3506e1742eff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:52.012158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e8092-55f3-4a9c-8d99-3506e1742eff service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:52.069502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 818/1636] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:08:53.343316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105846c-148e-4aaf-8a8a-de35ab2b1d0e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:08:53.343316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105846c-148e-4aaf-8a8a-de35ab2b1d0e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:08:53.343316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8402ea-d748-4688-b04c-9c49b877f4da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:08:53.343316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8402ea-d748-4688-b04c-9c49b877f4da service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:08:53.352881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8402ea-d748-4688-b04c-9c49b877f4da service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:08:53.352881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8402ea-d748-4688-b04c-9c49b877f4da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:08:53.410804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8402ea-d748-4688-b04c-9c49b877f4da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:08:53.412591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8402ea-d748-4688-b04c-9c49b877f4da service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:08:53.423727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 819/1637] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:08:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:00.344490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f11745-cb11-4a36-b4a1-dc401773b595 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:00.345038 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f11745-cb11-4a36-b4a1-dc401773b595 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:00.347209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f11745-cb11-4a36-b4a1-dc401773b595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:00.452875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-06f11745-cb11-4a36-b4a1-dc401773b595 None None] MFA Rules not processed for user `ec40d2360ab2417bb2d50414575d59d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:00.454380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-06f11745-cb11-4a36-b4a1-dc401773b595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:00.648636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 819/1638] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:00.650277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902c4e-f547-40fd-974a-6f7fc356e523 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:00.651154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902c4e-f547-40fd-974a-6f7fc356e523 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:00.651642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902c4e-f547-40fd-974a-6f7fc356e523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:00.687867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c9902c4e-f547-40fd-974a-6f7fc356e523 None None] MFA Rules not processed for user `ec40d2360ab2417bb2d50414575d59d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:00.689539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c9902c4e-f547-40fd-974a-6f7fc356e523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:00.707541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 820/1639] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:00.719605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f11745-cb11-4a36-b4a1-dc401773b595 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:00.746390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f11745-cb11-4a36-b4a1-dc401773b595 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:00.752324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73581f62-22b8-4f18-a3f1-73b8f635b561 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:00.752961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73581f62-22b8-4f18-a3f1-73b8f635b561 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:00.753241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73581f62-22b8-4f18-a3f1-73b8f635b561 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:00.753515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73581f62-22b8-4f18-a3f1-73b8f635b561 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:00.770326 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73581f62-22b8-4f18-a3f1-73b8f635b561 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:00.771936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73581f62-22b8-4f18-a3f1-73b8f635b561 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:00.817464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 820/1640] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:01.401978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902c4e-f547-40fd-974a-6f7fc356e523 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:01.577867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902c4e-f547-40fd-974a-6f7fc356e523 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:01.590708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:01.591357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:01.591633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:01.591984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] PATH_INFO: `/v3/users/491f22a4783f48d298a2507a0b10ddbd` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:01.593152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=491f22a4783f48d298a2507a0b10ddbd)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:01.595076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:01.747196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd999c5-079c-4387-a454-1b1dae1c7c17 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:01.748276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd999c5-079c-4387-a454-1b1dae1c7c17 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:01.748918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd999c5-079c-4387-a454-1b1dae1c7c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:01.854349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3cd999c5-079c-4387-a454-1b1dae1c7c17 None None] MFA Rules not processed for user `2d5e7d5c77084705b1f39406293ab585`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:01.856164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3cd999c5-079c-4387-a454-1b1dae1c7c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:01.874990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:01.951002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '491f22a4783f48d298a2507a0b10ddbd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:02.026335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '491f22a4783f48d298a2507a0b10ddbd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:02.037125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '491f22a4783f48d298a2507a0b10ddbd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:02.038342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '491f22a4783f48d298a2507a0b10ddbd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:02.039352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '491f22a4783f48d298a2507a0b10ddbd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:02.076480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 821/1641] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:02.087608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 821/1642] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:01 2022] DELETE /identity/v3/users/491f22a4783f48d298a2507a0b10ddbd => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:02.093047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b826c9a-6290-4169-8211-65758b0563f8 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:02.093338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b826c9a-6290-4169-8211-65758b0563f8 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:02.093617 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b826c9a-6290-4169-8211-65758b0563f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:02.095527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:02.097243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:02.129875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:02.153046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9b826c9a-6290-4169-8211-65758b0563f8 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:02.154390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9b826c9a-6290-4169-8211-65758b0563f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:02.281400 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5579bab-3e8e-4e0d-994a-439b3b12a7ea admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:02.297501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:02.299701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:02.300820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:02.310159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:02.317378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:02.333415 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 822/1643] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:02.334875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630593c-0bae-45fa-9ef3-11f5aaa3d05e None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:02.335225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630593c-0bae-45fa-9ef3-11f5aaa3d05e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:02.335607 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630593c-0bae-45fa-9ef3-11f5aaa3d05e None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:02.345663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 823/1644] 10.222.0.50 () {54 vars in 912 bytes} [Mon Aug 8 18:09:02 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:09:02.348168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3ef60-8416-4d08-a52d-cd512836f0be None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:02.348781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3ef60-8416-4d08-a52d-cd512836f0be None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:02.350962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3ef60-8416-4d08-a52d-cd512836f0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:02.436572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-acd3ef60-8416-4d08-a52d-cd512836f0be None None] MFA Rules not processed for user `95f02466042946ee96e37c5adc27b01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:02.438280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-acd3ef60-8416-4d08-a52d-cd512836f0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:02.460967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:02.462790 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:02.488235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 822/1645] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:02.489567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:02.495003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:02.576496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 824/1646] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:09:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:02.615825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:02.632000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a1904-d45f-4fd9-8be9-a9029ffbc28d service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:02.637920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3ef60-8416-4d08-a52d-cd512836f0be None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:02.649223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3ef60-8416-4d08-a52d-cd512836f0be None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:02.653066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c28c36-bad4-4d12-aed8-d3718c2a4e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:02.660418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c28c36-bad4-4d12-aed8-d3718c2a4e3e service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:02.662660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c28c36-bad4-4d12-aed8-d3718c2a4e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:02.666756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c28c36-bad4-4d12-aed8-d3718c2a4e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:02.668701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62d540-894d-4919-b8b6-4ae09429ee2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:02.670474 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62d540-894d-4919-b8b6-4ae09429ee2b service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:02.672736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62d540-894d-4919-b8b6-4ae09429ee2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:02.673238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62d540-894d-4919-b8b6-4ae09429ee2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:02.696867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62d540-894d-4919-b8b6-4ae09429ee2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:02.697020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c28c36-bad4-4d12-aed8-d3718c2a4e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:02.701103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62d540-894d-4919-b8b6-4ae09429ee2b service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:02.702279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c28c36-bad4-4d12-aed8-d3718c2a4e3e service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:02.736714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 825/1647] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:02.739693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 823/1648] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:04.579764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfdbdf-fdb0-49a1-a0c8-a73edf02100c None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:04.580259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfdbdf-fdb0-49a1-a0c8-a73edf02100c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:04.580394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfdbdf-fdb0-49a1-a0c8-a73edf02100c None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:04.582925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 826/1649] 10.222.0.50 () {54 vars in 961 bytes} [Mon Aug 8 18:09:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:09:04.590641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e5750-7f4d-4fc1-bf48-daf86f1d0cf3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:04.590641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e5750-7f4d-4fc1-bf48-daf86f1d0cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:04.590641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e5750-7f4d-4fc1-bf48-daf86f1d0cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:04.624568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fd5e5750-7f4d-4fc1-bf48-daf86f1d0cf3 None None] MFA Rules not processed for user `b81bdfeedcba4b0c93f13d14318b2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:04.638996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fd5e5750-7f4d-4fc1-bf48-daf86f1d0cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:04.743151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 824/1650] 10.222.0.50 () {58 vars in 1058 bytes} [Mon Aug 8 18:09:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:04.766896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfdbdf-fdb0-49a1-a0c8-a73edf02100c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:04.796204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfdbdf-fdb0-49a1-a0c8-a73edf02100c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:04.809492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:04.810641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:04.812453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:04.813033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:04.819844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:04.849570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:04.852445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:04.881433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 827/1651] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:09:05.164568 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec681def-87f9-4ba2-ae7b-66c3225668b8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:05.164831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec681def-87f9-4ba2-ae7b-66c3225668b8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:05.165088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec681def-87f9-4ba2-ae7b-66c3225668b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:05.250719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ec681def-87f9-4ba2-ae7b-66c3225668b8 None None] MFA Rules not processed for user `caf8d88ca4da4515bd9f1bec64da3375`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:05.252210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ec681def-87f9-4ba2-ae7b-66c3225668b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:05.319328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:05.320870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:05.362932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7b82d-83d7-4bcb-a5fc-edef37f8d415 service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:05.384772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:05.387598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:05.387927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:05.388204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] PATH_INFO: `/v3/projects/e023a8ae7a5340f2b652682525ac8a30` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:05.396590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e023a8ae7a5340f2b652682525ac8a30)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:05.401059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:05.528237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 825/1652] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:05.529603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14827330-e099-425a-81eb-f255bb0e7975 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:05.530643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14827330-e099-425a-81eb-f255bb0e7975 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:05.530754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14827330-e099-425a-81eb-f255bb0e7975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:05.675194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-14827330-e099-425a-81eb-f255bb0e7975 None None] MFA Rules not processed for user `4c7a6c1d08554db09d65394ec338f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:05.676286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-14827330-e099-425a-81eb-f255bb0e7975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:05.688783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e023a8ae7a5340f2b652682525ac8a30', 'request_id': 'req-6b5ff413-8a61-4c25-b651-2d605a02f6a1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:05.721040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e023a8ae7a5340f2b652682525ac8a30', 'request_id': 'req-6b5ff413-8a61-4c25-b651-2d605a02f6a1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:05.802573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e023a8ae7a5340f2b652682525ac8a30', 'request_id': 'req-6b5ff413-8a61-4c25-b651-2d605a02f6a1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:05.816052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:05.821251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] The token cache is being invalidate because project e023a8ae7a5340f2b652682525ac8a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:05.821646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:05.822318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:05.835986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 826/1654] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:05.835986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14827330-e099-425a-81eb-f255bb0e7975 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:05.848878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:05.867832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 828/1654] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:05 2022] DELETE /identity/v3/projects/e023a8ae7a5340f2b652682525ac8a30 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:05.869060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:05.902682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:05.987944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14827330-e099-425a-81eb-f255bb0e7975 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:06.002795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c2bcb-5991-4bd7-8678-7b7ef3fe0429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:06.003421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c2bcb-5991-4bd7-8678-7b7ef3fe0429 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:06.003704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c2bcb-5991-4bd7-8678-7b7ef3fe0429 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:06.003969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c2bcb-5991-4bd7-8678-7b7ef3fe0429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:06.007559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c10c2bcb-5991-4bd7-8678-7b7ef3fe0429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:06.039632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ff413-8a61-4c25-b651-2d605a02f6a1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:06.064148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:06.066620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:06.067005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:06.067284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] PATH_INFO: `/v3/users/c9cee55af1bb4dd7a7f7956ef088d59f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:06.068736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9cee55af1bb4dd7a7f7956ef088d59f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:06.069827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: WARNING py.warnings [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:09:06.069827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: warnings.warn(msg) Aug 08 18:09:06.069827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: Aug 08 18:09:06.075935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:06.208929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c2bcb-5991-4bd7-8678-7b7ef3fe0429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:06.211070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c2bcb-5991-4bd7-8678-7b7ef3fe0429 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:06.226158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 827/1655] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:06.227969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6942-ca2a-4f29-a671-0e81f4e3e81b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:06.228630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6942-ca2a-4f29-a671-0e81f4e3e81b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:06.229009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6942-ca2a-4f29-a671-0e81f4e3e81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:06.260859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f4eb6942-ca2a-4f29-a671-0e81f4e3e81b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:06.262147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f4eb6942-ca2a-4f29-a671-0e81f4e3e81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:06.275067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 828/1656] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:06.277865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48f674-9cbf-4a70-8eb1-7c29f8f652c3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:06.278293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48f674-9cbf-4a70-8eb1-7c29f8f652c3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:06.278712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48f674-9cbf-4a70-8eb1-7c29f8f652c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:06.354180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1c48f674-9cbf-4a70-8eb1-7c29f8f652c3 None None] MFA Rules not processed for user `4c7a6c1d08554db09d65394ec338f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:06.359493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9cee55af1bb4dd7a7f7956ef088d59f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:06.359769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1c48f674-9cbf-4a70-8eb1-7c29f8f652c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:06.376914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 829/1657] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:06.385254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f725b2-569d-4126-886e-cb96f521f0cb None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:06.385543 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f725b2-569d-4126-886e-cb96f521f0cb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:06.403669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f725b2-569d-4126-886e-cb96f521f0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:06.474490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-10f725b2-569d-4126-886e-cb96f521f0cb None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:06.475736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-10f725b2-569d-4126-886e-cb96f521f0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:06.515944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 830/1658] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:06.515944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f725b2-569d-4126-886e-cb96f521f0cb None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:06.518878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-10f725b2-569d-4126-886e-cb96f521f0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:06.554499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f725b2-569d-4126-886e-cb96f521f0cb None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:06.565161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9cee55af1bb4dd7a7f7956ef088d59f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:06.583777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9cee55af1bb4dd7a7f7956ef088d59f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:06.584698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc3dcb-8d8e-46be-b001-448581d74bd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:06.585923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9cee55af1bb4dd7a7f7956ef088d59f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:06.588372 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9cee55af1bb4dd7a7f7956ef088d59f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:06.590292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc3dcb-8d8e-46be-b001-448581d74bd7 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:06.590613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc3dcb-8d8e-46be-b001-448581d74bd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:06.590883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc3dcb-8d8e-46be-b001-448581d74bd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:06.598853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-adcc3dcb-8d8e-46be-b001-448581d74bd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:06.654791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc3dcb-8d8e-46be-b001-448581d74bd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:06.656433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc3dcb-8d8e-46be-b001-448581d74bd7 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:06.659820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 829/1659] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:05 2022] DELETE /identity/v3/users/c9cee55af1bb4dd7a7f7956ef088d59f => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:06.661456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:06.666286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:06.694159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 831/1660] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:06.697109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324f4f-75bd-45dc-bc94-a01a84871487 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:06.700781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324f4f-75bd-45dc-bc94-a01a84871487 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:06.701213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324f4f-75bd-45dc-bc94-a01a84871487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:06.746960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-84324f4f-75bd-45dc-bc94-a01a84871487 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:06.750764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-84324f4f-75bd-45dc-bc94-a01a84871487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:06.881347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5a091-30ab-4dd1-9be1-4bb8126f3d67 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:06.893232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff1763-10b5-4bf7-a3e6-c8716a223f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:06.894585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff1763-10b5-4bf7-a3e6-c8716a223f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:06.895016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff1763-10b5-4bf7-a3e6-c8716a223f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:06.895525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff1763-10b5-4bf7-a3e6-c8716a223f32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:06.896491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff1763-10b5-4bf7-a3e6-c8716a223f32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:09:06.896961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff1763-10b5-4bf7-a3e6-c8716a223f32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:06.904179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff1763-10b5-4bf7-a3e6-c8716a223f32 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:06.930565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 832/1661] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:06.942958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84324f4f-75bd-45dc-bc94-a01a84871487 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:06.944791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-84324f4f-75bd-45dc-bc94-a01a84871487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:06.957808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 830/1662] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:09:06.960284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff1763-10b5-4bf7-a3e6-c8716a223f32 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:06.965298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:06.998002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff1763-10b5-4bf7-a3e6-c8716a223f32 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:07.031715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398cfac-449b-460e-ac82-f0cabfe7dccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:07.031715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398cfac-449b-460e-ac82-f0cabfe7dccb admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:07.031715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398cfac-449b-460e-ac82-f0cabfe7dccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:07.031715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398cfac-449b-460e-ac82-f0cabfe7dccb admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:07.032172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398cfac-449b-460e-ac82-f0cabfe7dccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:07.033094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398cfac-449b-460e-ac82-f0cabfe7dccb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:07.062789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:07.095722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84324f4f-75bd-45dc-bc94-a01a84871487 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:07.106626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59bc14-10c3-4477-8f26-8a997aa02254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:07.107268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59bc14-10c3-4477-8f26-8a997aa02254 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:07.110289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59bc14-10c3-4477-8f26-8a997aa02254 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:07.110578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59bc14-10c3-4477-8f26-8a997aa02254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:07.135069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59bc14-10c3-4477-8f26-8a997aa02254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:07.137716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59bc14-10c3-4477-8f26-8a997aa02254 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:07.163391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 833/1663] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:09:07.211323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 831/1664] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:06 2022] POST /identity/v3/projects => generated 363 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:07.215441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59bc14-10c3-4477-8f26-8a997aa02254 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:07.232364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59bc14-10c3-4477-8f26-8a997aa02254 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:07.383095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a0ef8-bfd0-4bcf-982e-f6b9626e583b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:07.383762 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a0ef8-bfd0-4bcf-982e-f6b9626e583b admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:07.384392 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a0ef8-bfd0-4bcf-982e-f6b9626e583b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:07.384392 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a0ef8-bfd0-4bcf-982e-f6b9626e583b admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:07.384981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a0ef8-bfd0-4bcf-982e-f6b9626e583b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:07.395720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a0ef8-bfd0-4bcf-982e-f6b9626e583b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:07.535985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 834/1665] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:07 2022] POST /identity/v3/users => generated 282 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:07.540613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398cfac-449b-460e-ac82-f0cabfe7dccb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:07.578254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398cfac-449b-460e-ac82-f0cabfe7dccb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:07.597210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d586581-0e3c-46bb-b4ae-de3948bf6a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:07.597852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d586581-0e3c-46bb-b4ae-de3948bf6a68 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:07.598232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d586581-0e3c-46bb-b4ae-de3948bf6a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:07.599107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d586581-0e3c-46bb-b4ae-de3948bf6a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:07.601682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d586581-0e3c-46bb-b4ae-de3948bf6a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:07.606288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d586581-0e3c-46bb-b4ae-de3948bf6a68 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:07.640170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 832/1666] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:07.645459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a0ef8-bfd0-4bcf-982e-f6b9626e583b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:07.664485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a0ef8-bfd0-4bcf-982e-f6b9626e583b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:07.680066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d9b3-81f0-48f8-be42-93727cad8dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:07.680823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d9b3-81f0-48f8-be42-93727cad8dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:07.681169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d9b3-81f0-48f8-be42-93727cad8dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:07.683251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d9b3-81f0-48f8-be42-93727cad8dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:07.684170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d9b3-81f0-48f8-be42-93727cad8dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:07.690264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d9b3-81f0-48f8-be42-93727cad8dd6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:07.716734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d586581-0e3c-46bb-b4ae-de3948bf6a68 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:07.725320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 835/1667] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:07.737151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d586581-0e3c-46bb-b4ae-de3948bf6a68 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:07.758842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599026e-2b3a-49f4-852a-fa9897af7512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:07.759730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599026e-2b3a-49f4-852a-fa9897af7512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:07.762161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599026e-2b3a-49f4-852a-fa9897af7512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:07.762481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599026e-2b3a-49f4-852a-fa9897af7512 admin admin] PATH_INFO: `/v3/projects/5e4bf79da0f345a5a8a5a7565a08890e/users/37b4e16ac3f74fde82ebe5ed27662767/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:07.816120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599026e-2b3a-49f4-852a-fa9897af7512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4bf79da0f345a5a8a5a7565a08890e, user_id=37b4e16ac3f74fde82ebe5ed27662767, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:07.823256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599026e-2b3a-49f4-852a-fa9897af7512 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:07.958098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 833/1668] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:07 2022] PUT /identity/v3/projects/5e4bf79da0f345a5a8a5a7565a08890e/users/37b4e16ac3f74fde82ebe5ed27662767/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:07.958756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f789-191f-4785-ae36-b3e22c63ad0e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:07.959150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f789-191f-4785-ae36-b3e22c63ad0e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:07.959521 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f789-191f-4785-ae36-b3e22c63ad0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:08.049447 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3474f789-191f-4785-ae36-b3e22c63ad0e None None] MFA Rules not processed for user `37b4e16ac3f74fde82ebe5ed27662767`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:08.051273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3474f789-191f-4785-ae36-b3e22c63ad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:08.229355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599026e-2b3a-49f4-852a-fa9897af7512 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:08.231378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 836/1669] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 08 18:09:08.448019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599026e-2b3a-49f4-852a-fa9897af7512 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:08.466939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f995c87-5267-4018-9dab-2f12b8a972bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:08.467594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f995c87-5267-4018-9dab-2f12b8a972bd admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:08.467856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f995c87-5267-4018-9dab-2f12b8a972bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:08.468124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f995c87-5267-4018-9dab-2f12b8a972bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:08.468787 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f995c87-5267-4018-9dab-2f12b8a972bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:08.470723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f995c87-5267-4018-9dab-2f12b8a972bd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:08.655071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 834/1670] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:08 2022] POST /identity/v3/projects => generated 365 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:08.659172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474f789-191f-4785-ae36-b3e22c63ad0e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:08.676308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeec602-c835-4966-8210-20ec898f821f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:08.677406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeec602-c835-4966-8210-20ec898f821f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:08.677683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeec602-c835-4966-8210-20ec898f821f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:08.692824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474f789-191f-4785-ae36-b3e22c63ad0e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:08.741654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ebeec602-c835-4966-8210-20ec898f821f None None] MFA Rules not processed for user `2ec53f935a6c41f2af67e9fd2e13e12c`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:08.743306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ebeec602-c835-4966-8210-20ec898f821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:08.769067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:08.888989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 835/1671] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:08.909432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeec602-c835-4966-8210-20ec898f821f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:08.912291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307d34b-3bbe-46be-9904-d0a251de37e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:08.916101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307d34b-3bbe-46be-9904-d0a251de37e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:08.916652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307d34b-3bbe-46be-9904-d0a251de37e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:08.917147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ebeec602-c835-4966-8210-20ec898f821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:08.918097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307d34b-3bbe-46be-9904-d0a251de37e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:08.918955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307d34b-3bbe-46be-9904-d0a251de37e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:08.926856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307d34b-3bbe-46be-9904-d0a251de37e5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:08.998419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:09.062704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 837/1672] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:08 2022] POST /identity/v3/users => generated 289 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:09.065656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b82e2-8cd9-41ca-9d5e-9467ea154086 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:09.069688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b82e2-8cd9-41ca-9d5e-9467ea154086 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:09.070228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b82e2-8cd9-41ca-9d5e-9467ea154086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:09.087173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeec602-c835-4966-8210-20ec898f821f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:09.100720 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6500379-5183-4609-804c-b7f115ccf8a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:09.103788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6500379-5183-4609-804c-b7f115ccf8a7 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:09.104295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6500379-5183-4609-804c-b7f115ccf8a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:09.104859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6500379-5183-4609-804c-b7f115ccf8a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:09.123243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6500379-5183-4609-804c-b7f115ccf8a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:09.125192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6500379-5183-4609-804c-b7f115ccf8a7 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:09.149515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 836/1673] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:09:09.151057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6500379-5183-4609-804c-b7f115ccf8a7 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:09.153920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1c6b82e2-8cd9-41ca-9d5e-9467ea154086 None None] MFA Rules not processed for user `4c7a6c1d08554db09d65394ec338f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:09.160962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1c6b82e2-8cd9-41ca-9d5e-9467ea154086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:09.179439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6500379-5183-4609-804c-b7f115ccf8a7 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:09.186552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bef5b-ed9c-445b-8a7f-0f6f5cd4059e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:09.189014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bef5b-ed9c-445b-8a7f-0f6f5cd4059e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:09.189401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bef5b-ed9c-445b-8a7f-0f6f5cd4059e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:09.189770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bef5b-ed9c-445b-8a7f-0f6f5cd4059e admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:09.191811 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bef5b-ed9c-445b-8a7f-0f6f5cd4059e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:09.194167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bef5b-ed9c-445b-8a7f-0f6f5cd4059e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:09.222457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 837/1674] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:09.223537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bef5b-ed9c-445b-8a7f-0f6f5cd4059e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:09.243865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bef5b-ed9c-445b-8a7f-0f6f5cd4059e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:09.267647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94003a-6537-4500-b1bd-2b8e1f203cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:09.271090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94003a-6537-4500-b1bd-2b8e1f203cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:09.271090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94003a-6537-4500-b1bd-2b8e1f203cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:09.271090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94003a-6537-4500-b1bd-2b8e1f203cb1 admin admin] PATH_INFO: `/v3/projects/c3f682e9c9944f23a5c3b54fa69e6599/users/5f85ee2020914f988fc8fdc22680d911/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:09.314500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 838/1675] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:09.331455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94003a-6537-4500-b1bd-2b8e1f203cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f682e9c9944f23a5c3b54fa69e6599, user_id=5f85ee2020914f988fc8fdc22680d911, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:09.336810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94003a-6537-4500-b1bd-2b8e1f203cb1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:09.515769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 838/1676] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:09 2022] PUT /identity/v3/projects/c3f682e9c9944f23a5c3b54fa69e6599/users/5f85ee2020914f988fc8fdc22680d911/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:09.519891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b82e2-8cd9-41ca-9d5e-9467ea154086 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:09.539237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b82e2-8cd9-41ca-9d5e-9467ea154086 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:09.741865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db46a2-9280-4a1a-b580-10076a53ede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:09.750304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db46a2-9280-4a1a-b580-10076a53ede4 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:09.750591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db46a2-9280-4a1a-b580-10076a53ede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:09.750864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db46a2-9280-4a1a-b580-10076a53ede4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:09.751677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db46a2-9280-4a1a-b580-10076a53ede4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:09.753358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db46a2-9280-4a1a-b580-10076a53ede4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:09.780347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 839/1677] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:09.784869 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94003a-6537-4500-b1bd-2b8e1f203cb1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:09.797521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94003a-6537-4500-b1bd-2b8e1f203cb1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:09.823498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565a4e6-674d-4211-9039-faa1eb1e0397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:09.825104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565a4e6-674d-4211-9039-faa1eb1e0397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:09.825554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565a4e6-674d-4211-9039-faa1eb1e0397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:09.826886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565a4e6-674d-4211-9039-faa1eb1e0397 admin admin] PATH_INFO: `/v3/projects/c3f682e9c9944f23a5c3b54fa69e6599/users/5f85ee2020914f988fc8fdc22680d911/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:09.833498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565a4e6-674d-4211-9039-faa1eb1e0397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f682e9c9944f23a5c3b54fa69e6599, user_id=5f85ee2020914f988fc8fdc22680d911, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:09.836202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565a4e6-674d-4211-9039-faa1eb1e0397 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:09.882014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db46a2-9280-4a1a-b580-10076a53ede4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:09.884241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d7db46a2-9280-4a1a-b580-10076a53ede4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:09.921034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db46a2-9280-4a1a-b580-10076a53ede4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:09.921222 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 839/1678] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:09 2022] PUT /identity/v3/projects/c3f682e9c9944f23a5c3b54fa69e6599/users/5f85ee2020914f988fc8fdc22680d911/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:09.922684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d69f4-e82e-4acf-8170-67fa2bb5de22 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:09.922969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d69f4-e82e-4acf-8170-67fa2bb5de22 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:09.923240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d69f4-e82e-4acf-8170-67fa2bb5de22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:10.004460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-911d69f4-e82e-4acf-8170-67fa2bb5de22 None None] MFA Rules not processed for user `5f85ee2020914f988fc8fdc22680d911`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:10.005594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-911d69f4-e82e-4acf-8170-67fa2bb5de22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:10.048443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:10.082909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:10.083679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:10.085712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:10.091047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] PATH_INFO: `/v3/projects/3b463a1983da46d398345bee415f1a55` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:10.094973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b463a1983da46d398345bee415f1a55)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:10.097344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: WARNING py.warnings [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 18:09:10.097344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: warnings.warn(msg) Aug 08 18:09:10.097344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: Aug 08 18:09:10.100442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:10.209802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 840/1679] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:10.223500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d69f4-e82e-4acf-8170-67fa2bb5de22 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:10.225209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-911d69f4-e82e-4acf-8170-67fa2bb5de22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:10.386149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b463a1983da46d398345bee415f1a55', 'request_id': 'req-2eea21ad-d2a6-4981-bc47-80263e20850c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:10.387385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d69f4-e82e-4acf-8170-67fa2bb5de22 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:10.407064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b463a1983da46d398345bee415f1a55', 'request_id': 'req-2eea21ad-d2a6-4981-bc47-80263e20850c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:10.410791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:10.490273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b463a1983da46d398345bee415f1a55', 'request_id': 'req-2eea21ad-d2a6-4981-bc47-80263e20850c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:10.543359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] The token cache is being invalidate because project 3b463a1983da46d398345bee415f1a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:10.544357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f4108-a661-493c-8f69-bc37f31a658a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:10.546273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:10.547050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f4108-a661-493c-8f69-bc37f31a658a service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:10.547912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eea21ad-d2a6-4981-bc47-80263e20850c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:10.548291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f4108-a661-493c-8f69-bc37f31a658a service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:10.548973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f4108-a661-493c-8f69-bc37f31a658a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:10.553731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 840/1680] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:09 2022] DELETE /identity/v3/projects/3b463a1983da46d398345bee415f1a55 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:10.558444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7c06c-5178-45c4-9611-d4970fc16e92 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:10.558444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7c06c-5178-45c4-9611-d4970fc16e92 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:10.558444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7c06c-5178-45c4-9611-d4970fc16e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:10.558748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f58f4108-a661-493c-8f69-bc37f31a658a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:10.603848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9fe7c06c-5178-45c4-9611-d4970fc16e92 None None] MFA Rules not processed for user `2d5e7d5c77084705b1f39406293ab585`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:10.607067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9fe7c06c-5178-45c4-9611-d4970fc16e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:10.713440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f4108-a661-493c-8f69-bc37f31a658a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:10.715577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f4108-a661-493c-8f69-bc37f31a658a service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:10.738920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 841/1681] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:09:10.740461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841047-c4a0-4cbd-a261-45c3761ea1cd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:10.740612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841047-c4a0-4cbd-a261-45c3761ea1cd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:10.740876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841047-c4a0-4cbd-a261-45c3761ea1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:10.787989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-94841047-c4a0-4cbd-a261-45c3761ea1cd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:10.789495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-94841047-c4a0-4cbd-a261-45c3761ea1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:10.896139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 841/1682] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:11.038871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 842/1683] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:11.048540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10605fd-0d04-4f09-b38b-243217c1530a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:11.048540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10605fd-0d04-4f09-b38b-243217c1530a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:11.048540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10605fd-0d04-4f09-b38b-243217c1530a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:11.102566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c10605fd-0d04-4f09-b38b-243217c1530a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:11.107385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c10605fd-0d04-4f09-b38b-243217c1530a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:11.158356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94841047-c4a0-4cbd-a261-45c3761ea1cd None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:11.160062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-94841047-c4a0-4cbd-a261-45c3761ea1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:11.162694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 842/1684] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:09:11.166969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10605fd-0d04-4f09-b38b-243217c1530a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:11.186996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94841047-c4a0-4cbd-a261-45c3761ea1cd None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:11.196058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f631d2-2822-4182-a989-2a0aecd833ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:11.197060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f631d2-2822-4182-a989-2a0aecd833ff service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:11.199331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f631d2-2822-4182-a989-2a0aecd833ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:11.199468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f631d2-2822-4182-a989-2a0aecd833ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:11.202617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10605fd-0d04-4f09-b38b-243217c1530a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:11.206110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a0f631d2-2822-4182-a989-2a0aecd833ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:11.225282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5653443-85a4-47e9-9273-317a54decbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:11.241481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5653443-85a4-47e9-9273-317a54decbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:11.243723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5653443-85a4-47e9-9273-317a54decbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:11.244174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5653443-85a4-47e9-9273-317a54decbd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:11.245049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5653443-85a4-47e9-9273-317a54decbd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:09:11.250168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5653443-85a4-47e9-9273-317a54decbd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:11.258128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5653443-85a4-47e9-9273-317a54decbd5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:11.285123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 843/1685] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:09:11.289894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5653443-85a4-47e9-9273-317a54decbd5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:11.316947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5653443-85a4-47e9-9273-317a54decbd5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:11.325977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0971a8d-4c94-427a-8158-35b64e4ad73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:11.327036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0971a8d-4c94-427a-8158-35b64e4ad73e admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:11.327374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0971a8d-4c94-427a-8158-35b64e4ad73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:11.327999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0971a8d-4c94-427a-8158-35b64e4ad73e admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:11.328475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0971a8d-4c94-427a-8158-35b64e4ad73e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:11.331294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0971a8d-4c94-427a-8158-35b64e4ad73e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:11.357591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f631d2-2822-4182-a989-2a0aecd833ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:11.359051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f631d2-2822-4182-a989-2a0aecd833ff service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:11.362976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:11.380082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 843/1686] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:09:11.380082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5040e7c-24bf-4a26-a693-ea4cec697a97 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:11.380082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5040e7c-24bf-4a26-a693-ea4cec697a97 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:11.380082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5040e7c-24bf-4a26-a693-ea4cec697a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:11.520747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a5040e7c-24bf-4a26-a693-ea4cec697a97 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:11.522336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a5040e7c-24bf-4a26-a693-ea4cec697a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:11.530732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 844/1687] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:11 2022] POST /identity/v3/projects => generated 405 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:11.535460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0971a8d-4c94-427a-8158-35b64e4ad73e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:11.709225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 844/1688] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:09:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:11.747579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:11.828328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0971a8d-4c94-427a-8158-35b64e4ad73e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:11.838520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d16d7a-aca9-4431-8932-e9e4eeb31f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:11.839759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d16d7a-aca9-4431-8932-e9e4eeb31f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:11.840030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d16d7a-aca9-4431-8932-e9e4eeb31f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:11.840456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d16d7a-aca9-4431-8932-e9e4eeb31f00 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:11.841418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d16d7a-aca9-4431-8932-e9e4eeb31f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:11.843622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d16d7a-aca9-4431-8932-e9e4eeb31f00 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:11.989596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 845/1689] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:11 2022] POST /identity/v3/users => generated 303 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:11.993180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5040e7c-24bf-4a26-a693-ea4cec697a97 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:12.020893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5040e7c-24bf-4a26-a693-ea4cec697a97 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:12.057182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee6805-98fe-4861-a24c-275512a41363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:12.057841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee6805-98fe-4861-a24c-275512a41363 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:12.060276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee6805-98fe-4861-a24c-275512a41363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:12.063940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee6805-98fe-4861-a24c-275512a41363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:12.066328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee6805-98fe-4861-a24c-275512a41363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:12.067955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee6805-98fe-4861-a24c-275512a41363 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:12.110093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 845/1690] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:12.113530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d16d7a-aca9-4431-8932-e9e4eeb31f00 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:12.125117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d16d7a-aca9-4431-8932-e9e4eeb31f00 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:12.133206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2138bb1-d614-4488-ae63-5c8a49db9f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:12.134196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2138bb1-d614-4488-ae63-5c8a49db9f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:12.134829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2138bb1-d614-4488-ae63-5c8a49db9f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:12.135251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2138bb1-d614-4488-ae63-5c8a49db9f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:12.136860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2138bb1-d614-4488-ae63-5c8a49db9f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:12.184056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2138bb1-d614-4488-ae63-5c8a49db9f39 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:12.238647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 846/1691] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:12.240147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee6805-98fe-4861-a24c-275512a41363 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:12.252347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee6805-98fe-4861-a24c-275512a41363 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:12.268600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae6667-9b72-4fa7-8ee4-52efdc9fb2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:12.269526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae6667-9b72-4fa7-8ee4-52efdc9fb2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:12.269967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae6667-9b72-4fa7-8ee4-52efdc9fb2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:12.270489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae6667-9b72-4fa7-8ee4-52efdc9fb2e4 admin admin] PATH_INFO: `/v3/projects/eeb767c5e2ba406db2cfe3ffb22f0198/users/18d8cd35af894368bef309092d55cb49/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:12.289611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2138bb1-d614-4488-ae63-5c8a49db9f39 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:12.294153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b2138bb1-d614-4488-ae63-5c8a49db9f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:12.316755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae6667-9b72-4fa7-8ee4-52efdc9fb2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb767c5e2ba406db2cfe3ffb22f0198, user_id=18d8cd35af894368bef309092d55cb49, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:12.319460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae6667-9b72-4fa7-8ee4-52efdc9fb2e4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:12.400792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 846/1692] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:12 2022] PUT /identity/v3/projects/eeb767c5e2ba406db2cfe3ffb22f0198/users/18d8cd35af894368bef309092d55cb49/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:09:12.418855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211adde2-6722-484a-9411-f0d13a3fa1d1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:12.419140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211adde2-6722-484a-9411-f0d13a3fa1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:12.419414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211adde2-6722-484a-9411-f0d13a3fa1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:12.482959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-211adde2-6722-484a-9411-f0d13a3fa1d1 None None] MFA Rules not processed for user `18d8cd35af894368bef309092d55cb49`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:12.484384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-211adde2-6722-484a-9411-f0d13a3fa1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:12.612803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2138bb1-d614-4488-ae63-5c8a49db9f39 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:12.653762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 847/1693] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:12 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:12.656822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124b1ab-c94d-4cda-a797-ab6bbb57dc94 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:12.659389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124b1ab-c94d-4cda-a797-ab6bbb57dc94 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:12.659803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124b1ab-c94d-4cda-a797-ab6bbb57dc94 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:12.661572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 848/1694] 10.222.0.50 () {54 vars in 912 bytes} [Mon Aug 8 18:09:12 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:09:12.665174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124b1ab-c94d-4cda-a797-ab6bbb57dc94 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:12.723322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:12.772281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d477ceb-f81c-4581-8965-c00492430c34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:12.773457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d477ceb-f81c-4581-8965-c00492430c34 service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:12.773740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d477ceb-f81c-4581-8965-c00492430c34 service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:12.774012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d477ceb-f81c-4581-8965-c00492430c34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:12.779432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124b1ab-c94d-4cda-a797-ab6bbb57dc94 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:12.786804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6fd08-aeaf-4a5f-91fb-98454b9724ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:12.787552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6fd08-aeaf-4a5f-91fb-98454b9724ac admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:12.787933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6fd08-aeaf-4a5f-91fb-98454b9724ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:12.788333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6fd08-aeaf-4a5f-91fb-98454b9724ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:12.790254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6fd08-aeaf-4a5f-91fb-98454b9724ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:12.791640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6fd08-aeaf-4a5f-91fb-98454b9724ac admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:12.879517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:12.904641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d477ceb-f81c-4581-8965-c00492430c34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:12.906374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d477ceb-f81c-4581-8965-c00492430c34 service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:12.928803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 849/1695] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:12 2022] POST /identity/v3/projects => generated 403 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:12.931017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba7538-de54-41f9-8f55-a9b7be0d008e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:12.933833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba7538-de54-41f9-8f55-a9b7be0d008e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:12.934393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba7538-de54-41f9-8f55-a9b7be0d008e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:12.938491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 847/1696] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:09:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 18:09:12.940100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d477ceb-f81c-4581-8965-c00492430c34 service swift] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:12.965778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d477ceb-f81c-4581-8965-c00492430c34 service swift] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:12.985775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-eeba7538-de54-41f9-8f55-a9b7be0d008e None None] MFA Rules not processed for user `95f02466042946ee96e37c5adc27b01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:12.990140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eeba7538-de54-41f9-8f55-a9b7be0d008e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:13.216454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ff1db-b05b-40b1-b830-b6641a2b4b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:13.217115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ff1db-b05b-40b1-b830-b6641a2b4b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:13.217394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ff1db-b05b-40b1-b830-b6641a2b4b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:13.217655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ff1db-b05b-40b1-b830-b6641a2b4b4f admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:13.218291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ff1db-b05b-40b1-b830-b6641a2b4b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:13.248297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ff1db-b05b-40b1-b830-b6641a2b4b4f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:13.260482 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 850/1697] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:09:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:13.373498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba7538-de54-41f9-8f55-a9b7be0d008e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:13.375376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eeba7538-de54-41f9-8f55-a9b7be0d008e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:13.393704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 848/1698] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:12 2022] POST /identity/v3/users => generated 308 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:13.402290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ff1db-b05b-40b1-b830-b6641a2b4b4f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:13.403597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba7538-de54-41f9-8f55-a9b7be0d008e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:13.444603 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ff0da-ad79-44b6-a85e-4d2a0d8a51f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:13.445116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff0da-ad79-44b6-a85e-4d2a0d8a51f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:13.445395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff0da-ad79-44b6-a85e-4d2a0d8a51f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:13.445650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff0da-ad79-44b6-a85e-4d2a0d8a51f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:13.452098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ff1db-b05b-40b1-b830-b6641a2b4b4f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:13.464949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546dfa3-b929-44e5-bbd3-e27afb8707da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:13.465632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546dfa3-b929-44e5-bbd3-e27afb8707da admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:13.465910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546dfa3-b929-44e5-bbd3-e27afb8707da admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:13.466175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546dfa3-b929-44e5-bbd3-e27afb8707da admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:13.466818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546dfa3-b929-44e5-bbd3-e27afb8707da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:13.468625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546dfa3-b929-44e5-bbd3-e27afb8707da admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:13.501368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ff0da-ad79-44b6-a85e-4d2a0d8a51f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:13.502694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 849/1699] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:09:13.508754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ff0da-ad79-44b6-a85e-4d2a0d8a51f0 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:13.518324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546dfa3-b929-44e5-bbd3-e27afb8707da admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:13.541128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546dfa3-b929-44e5-bbd3-e27afb8707da admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:13.548340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dd8e9-a688-4086-b721-22696b0610f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:13.549223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dd8e9-a688-4086-b721-22696b0610f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:13.549500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dd8e9-a688-4086-b721-22696b0610f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:13.550892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dd8e9-a688-4086-b721-22696b0610f8 admin admin] PATH_INFO: `/v3/projects/5cbb71de47674b29a55f771cb882509b/users/b19ae7463a974f62a34e5ee751bff6b3/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:13.572877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 851/1700] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:09:13.632781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dd8e9-a688-4086-b721-22696b0610f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbb71de47674b29a55f771cb882509b, user_id=b19ae7463a974f62a34e5ee751bff6b3, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:13.641762 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dd8e9-a688-4086-b721-22696b0610f8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:13.738967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ff0da-ad79-44b6-a85e-4d2a0d8a51f0 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:13.739131 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 850/1701] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:13 2022] PUT /identity/v3/projects/5cbb71de47674b29a55f771cb882509b/users/b19ae7463a974f62a34e5ee751bff6b3/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:13.868912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ff0da-ad79-44b6-a85e-4d2a0d8a51f0 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:13.961181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4bbb4-607c-464e-97cf-d34ad38b2b92 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:13.961464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4bbb4-607c-464e-97cf-d34ad38b2b92 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:13.962239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4bbb4-607c-464e-97cf-d34ad38b2b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:14.038881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef24e7-6a7b-489a-bebc-b0901c857c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:14.040569 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef24e7-6a7b-489a-bebc-b0901c857c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:14.043514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef24e7-6a7b-489a-bebc-b0901c857c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:14.044193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef24e7-6a7b-489a-bebc-b0901c857c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:14.045252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef24e7-6a7b-489a-bebc-b0901c857c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:14.057063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef24e7-6a7b-489a-bebc-b0901c857c3d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:14.063544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-dfd4bbb4-607c-464e-97cf-d34ad38b2b92 None None] MFA Rules not processed for user `95f02466042946ee96e37c5adc27b01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:14.065786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-dfd4bbb4-607c-464e-97cf-d34ad38b2b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:14.164076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 852/1702] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:09:14.170032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef24e7-6a7b-489a-bebc-b0901c857c3d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:14.205357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef24e7-6a7b-489a-bebc-b0901c857c3d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:14.243532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96571f2e-bf72-4539-bbad-86d933533b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:14.244095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 851/1703] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Aug 8 18:09:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:09:14.246120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96571f2e-bf72-4539-bbad-86d933533b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:14.246535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96571f2e-bf72-4539-bbad-86d933533b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:14.246891 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96571f2e-bf72-4539-bbad-86d933533b69 admin admin] PATH_INFO: `/v3/projects/5cbb71de47674b29a55f771cb882509b/users/b19ae7463a974f62a34e5ee751bff6b3/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:14.249339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96571f2e-bf72-4539-bbad-86d933533b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbb71de47674b29a55f771cb882509b, user_id=b19ae7463a974f62a34e5ee751bff6b3, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:14.256123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96571f2e-bf72-4539-bbad-86d933533b69 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:14.278692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 853/1704] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:14 2022] PUT /identity/v3/projects/5cbb71de47674b29a55f771cb882509b/users/b19ae7463a974f62a34e5ee751bff6b3/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:14.296554 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605d42a-628d-46b4-8b60-b65179a7588c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:14.303158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605d42a-628d-46b4-8b60-b65179a7588c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:14.303520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605d42a-628d-46b4-8b60-b65179a7588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:14.338155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96571f2e-bf72-4539-bbad-86d933533b69 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:14.341316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-96571f2e-bf72-4539-bbad-86d933533b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:14.400549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4605d42a-628d-46b4-8b60-b65179a7588c None None] MFA Rules not processed for user `b19ae7463a974f62a34e5ee751bff6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:14.402724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4605d42a-628d-46b4-8b60-b65179a7588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:14.454928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:14.527486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96571f2e-bf72-4539-bbad-86d933533b69 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:14.532053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81022125-9f5b-4993-8847-660ac9f255e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:14.532715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81022125-9f5b-4993-8847-660ac9f255e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:14.533093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81022125-9f5b-4993-8847-660ac9f255e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:14.533439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81022125-9f5b-4993-8847-660ac9f255e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:14.555817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81022125-9f5b-4993-8847-660ac9f255e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:14.563019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81022125-9f5b-4993-8847-660ac9f255e8 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:14.565203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:14.623262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 854/1705] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:09:14.659157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 852/1706] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:14.673737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81022125-9f5b-4993-8847-660ac9f255e8 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:14.907799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81022125-9f5b-4993-8847-660ac9f255e8 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:14.922801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca8fdd-bd44-4868-98b3-e4f8a3213018 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:14.923920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca8fdd-bd44-4868-98b3-e4f8a3213018 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:14.924308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca8fdd-bd44-4868-98b3-e4f8a3213018 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:14.925517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca8fdd-bd44-4868-98b3-e4f8a3213018 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:14.959967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca8fdd-bd44-4868-98b3-e4f8a3213018 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:14.964856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca8fdd-bd44-4868-98b3-e4f8a3213018 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:15.007176 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 855/1707] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:14 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:15.386279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605d42a-628d-46b4-8b60-b65179a7588c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:15.554108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605d42a-628d-46b4-8b60-b65179a7588c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:15.581685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:15.582821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:15.582821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:15.583005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] PATH_INFO: `/v3/users/18d8cd35af894368bef309092d55cb49` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:15.587002 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18d8cd35af894368bef309092d55cb49)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:15.596206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:15.966688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18d8cd35af894368bef309092d55cb49'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:16.079158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18d8cd35af894368bef309092d55cb49'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:16.083421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18d8cd35af894368bef309092d55cb49'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:16.084333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18d8cd35af894368bef309092d55cb49'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:16.085057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18d8cd35af894368bef309092d55cb49'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:16.116469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 853/1708] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:15 2022] DELETE /identity/v3/users/18d8cd35af894368bef309092d55cb49 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:16.121718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58204a32-3df1-43d5-9bd4-291b53c79f98 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:16.122274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58204a32-3df1-43d5-9bd4-291b53c79f98 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:16.122274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58204a32-3df1-43d5-9bd4-291b53c79f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:16.150969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-58204a32-3df1-43d5-9bd4-291b53c79f98 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:16.153033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-58204a32-3df1-43d5-9bd4-291b53c79f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:16.290158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 856/1709] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:16.435962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:16.445000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:16.664595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc40684-8736-45a5-a11d-83b55d06f807 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:16.681894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6762f-42d0-4e5a-ac4e-a4e0c8be523c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:16.685351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6762f-42d0-4e5a-ac4e-a4e0c8be523c service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:16.687059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6762f-42d0-4e5a-ac4e-a4e0c8be523c service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:16.687570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6762f-42d0-4e5a-ac4e-a4e0c8be523c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:16.723103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:16.730183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6762f-42d0-4e5a-ac4e-a4e0c8be523c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:16.735040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6762f-42d0-4e5a-ac4e-a4e0c8be523c service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:16.766062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 854/1710] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:17.468309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a44438-f765-4740-bc6d-e3408749da8c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:17.468912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a44438-f765-4740-bc6d-e3408749da8c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:17.472758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a44438-f765-4740-bc6d-e3408749da8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:17.500924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a5a44438-f765-4740-bc6d-e3408749da8c None None] MFA Rules not processed for user `caf8d88ca4da4515bd9f1bec64da3375`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:17.503260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a5a44438-f765-4740-bc6d-e3408749da8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:17.769343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 857/1711] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:18.819916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730c17d-24f9-4304-8c22-ca731f403432 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:18.820533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730c17d-24f9-4304-8c22-ca731f403432 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:18.820533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730c17d-24f9-4304-8c22-ca731f403432 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:18.821834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 855/1712] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:09:18.872911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84c58c-c881-43f8-a97a-b81393fe0198 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:18.873458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84c58c-c881-43f8-a97a-b81393fe0198 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:18.873458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84c58c-c881-43f8-a97a-b81393fe0198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:18.917718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0b84c58c-c881-43f8-a97a-b81393fe0198 None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:18.918964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0b84c58c-c881-43f8-a97a-b81393fe0198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:18.938238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:19.052422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 858/1713] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:09:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:19.386385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730c17d-24f9-4304-8c22-ca731f403432 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:19.388129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6730c17d-24f9-4304-8c22-ca731f403432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:19.411797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730c17d-24f9-4304-8c22-ca731f403432 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:19.428842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec163b8-5a97-46da-9bbc-a8a0a53097bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:19.430044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec163b8-5a97-46da-9bbc-a8a0a53097bd service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:19.430514 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec163b8-5a97-46da-9bbc-a8a0a53097bd service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:19.430942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec163b8-5a97-46da-9bbc-a8a0a53097bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:19.434200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84c58c-c881-43f8-a97a-b81393fe0198 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:19.436928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0b84c58c-c881-43f8-a97a-b81393fe0198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:19.447045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec163b8-5a97-46da-9bbc-a8a0a53097bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:19.450168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec163b8-5a97-46da-9bbc-a8a0a53097bd service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:19.460168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84c58c-c881-43f8-a97a-b81393fe0198 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:19.465291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 856/1714] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:19.484713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:19.485358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:19.485631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:19.485897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] PATH_INFO: `/v3/projects/eeb767c5e2ba406db2cfe3ffb22f0198` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:19.487487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeb767c5e2ba406db2cfe3ffb22f0198)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:19.489615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:19.578073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36196a87-ec63-4eba-96fa-add6d026b0ac None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:19.579330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36196a87-ec63-4eba-96fa-add6d026b0ac None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:19.579491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36196a87-ec63-4eba-96fa-add6d026b0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:19.616680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-36196a87-ec63-4eba-96fa-add6d026b0ac None None] MFA Rules not processed for user `2ec53f935a6c41f2af67e9fd2e13e12c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:19.619637 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-36196a87-ec63-4eba-96fa-add6d026b0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:19.645195 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeb767c5e2ba406db2cfe3ffb22f0198', 'request_id': 'req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:19.661931 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeb767c5e2ba406db2cfe3ffb22f0198', 'request_id': 'req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:19.729935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeb767c5e2ba406db2cfe3ffb22f0198', 'request_id': 'req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:19.747066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 857/1715] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:19.762162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] The token cache is being invalidate because project eeb767c5e2ba406db2cfe3ffb22f0198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:19.762162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:19.762162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:19.762162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 859/1716] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:19 2022] DELETE /identity/v3/projects/eeb767c5e2ba406db2cfe3ffb22f0198 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:19.765265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36196a87-ec63-4eba-96fa-add6d026b0ac None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:19.902368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36196a87-ec63-4eba-96fa-add6d026b0ac None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:19.909524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:19.910577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:19.910951 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:19.911322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] PATH_INFO: `/v3/users/b19ae7463a974f62a34e5ee751bff6b3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:19.912913 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b19ae7463a974f62a34e5ee751bff6b3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:19.915274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:20.203486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b19ae7463a974f62a34e5ee751bff6b3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:20.296602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b19ae7463a974f62a34e5ee751bff6b3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:20.308846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b19ae7463a974f62a34e5ee751bff6b3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:20.309616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b19ae7463a974f62a34e5ee751bff6b3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:20.312559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2eae0ba7-20a2-4275-a488-bf342bcedab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b19ae7463a974f62a34e5ee751bff6b3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:20.353686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 858/1717] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:19 2022] DELETE /identity/v3/users/b19ae7463a974f62a34e5ee751bff6b3 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:21.042281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:21.044992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:21.071585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f8e82-b6dd-4e47-80dd-beeb9983b1d8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:21.207994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:21.208673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:21.208950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:21.212925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] PATH_INFO: `/v3/projects/5cbb71de47674b29a55f771cb882509b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:21.215003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbb71de47674b29a55f771cb882509b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:21.217510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:21.373506 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:21.392158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbb71de47674b29a55f771cb882509b', 'request_id': 'req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:21.406207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbb71de47674b29a55f771cb882509b', 'request_id': 'req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:21.489280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbb71de47674b29a55f771cb882509b', 'request_id': 'req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:21.503536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] The token cache is being invalidate because project 5cbb71de47674b29a55f771cb882509b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:21.504447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:21.508695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-17ba75c7-2808-49e5-b1c4-f933f5aedc1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:21.516185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 860/1718] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:21 2022] DELETE /identity/v3/projects/5cbb71de47674b29a55f771cb882509b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:21.517548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a770c23-8bb9-4490-b2ba-80ea39815610 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:21.518011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a770c23-8bb9-4490-b2ba-80ea39815610 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:21.518136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a770c23-8bb9-4490-b2ba-80ea39815610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:21.536867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:21.546391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0a770c23-8bb9-4490-b2ba-80ea39815610 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:21.547702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0a770c23-8bb9-4490-b2ba-80ea39815610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:21.662407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 859/1719] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:21.670041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15dffb-e8f0-42fa-8722-adc8789b049d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:21.676858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15dffb-e8f0-42fa-8722-adc8789b049d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:21.678220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15dffb-e8f0-42fa-8722-adc8789b049d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:21.707957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cb15dffb-e8f0-42fa-8722-adc8789b049d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:21.709860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cb15dffb-e8f0-42fa-8722-adc8789b049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:21.732692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 861/1720] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:21.736201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a770c23-8bb9-4490-b2ba-80ea39815610 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:21.762088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a770c23-8bb9-4490-b2ba-80ea39815610 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:21.767586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaeaa8-f371-4a5f-862f-2895c18fdfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:21.768534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaeaa8-f371-4a5f-862f-2895c18fdfcb admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:21.768837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaeaa8-f371-4a5f-862f-2895c18fdfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:21.769109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaeaa8-f371-4a5f-862f-2895c18fdfcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:21.781636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaeaa8-f371-4a5f-862f-2895c18fdfcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:09:21.782329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaeaa8-f371-4a5f-862f-2895c18fdfcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:21.784275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaeaa8-f371-4a5f-862f-2895c18fdfcb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:21.818720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 860/1721] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:09:21.824254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15dffb-e8f0-42fa-8722-adc8789b049d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:21.839916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15dffb-e8f0-42fa-8722-adc8789b049d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:21.856794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf677b-416d-485d-8f0e-d7020a6c45bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:21.857748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf677b-416d-485d-8f0e-d7020a6c45bf admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:21.858199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf677b-416d-485d-8f0e-d7020a6c45bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:21.858658 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf677b-416d-485d-8f0e-d7020a6c45bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:21.859537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf677b-416d-485d-8f0e-d7020a6c45bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:21.861662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf677b-416d-485d-8f0e-d7020a6c45bf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:21.967059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 862/1722] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:21 2022] POST /identity/v3/projects => generated 403 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:21.972321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaeaa8-f371-4a5f-862f-2895c18fdfcb admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:21.985318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaeaa8-f371-4a5f-862f-2895c18fdfcb admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:22.120325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e21d1-e609-4664-adac-16aa949f5ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:22.120325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e21d1-e609-4664-adac-16aa949f5ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:22.120325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e21d1-e609-4664-adac-16aa949f5ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:22.120325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e21d1-e609-4664-adac-16aa949f5ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:22.120670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e21d1-e609-4664-adac-16aa949f5ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:22.120670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e21d1-e609-4664-adac-16aa949f5ee0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:22.263662 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf677b-416d-485d-8f0e-d7020a6c45bf admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:22.271668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 861/1723] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:21 2022] POST /identity/v3/users => generated 302 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:22.279718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf677b-416d-485d-8f0e-d7020a6c45bf admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:22.287306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df688aa-c499-414c-996f-e7d1bf551704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:22.288216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df688aa-c499-414c-996f-e7d1bf551704 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:22.288729 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df688aa-c499-414c-996f-e7d1bf551704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:22.289190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df688aa-c499-414c-996f-e7d1bf551704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:22.290038 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df688aa-c499-414c-996f-e7d1bf551704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:22.336851 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df688aa-c499-414c-996f-e7d1bf551704 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:22.368486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e21d1-e609-4664-adac-16aa949f5ee0 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:22.379443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 863/1724] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:22.381704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e21d1-e609-4664-adac-16aa949f5ee0 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:22.394235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17440b-8626-4e3e-a936-bfe7dc38703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:22.394864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17440b-8626-4e3e-a936-bfe7dc38703f admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:22.395144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17440b-8626-4e3e-a936-bfe7dc38703f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:22.395415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17440b-8626-4e3e-a936-bfe7dc38703f admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:22.400582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17440b-8626-4e3e-a936-bfe7dc38703f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:22.400582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17440b-8626-4e3e-a936-bfe7dc38703f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:22.417593 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df688aa-c499-414c-996f-e7d1bf551704 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:22.430187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 862/1725] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:22.434001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df688aa-c499-414c-996f-e7d1bf551704 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:22.444755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd30ce8-5496-46f4-a99b-6341945b7c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:22.445697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd30ce8-5496-46f4-a99b-6341945b7c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:22.446124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd30ce8-5496-46f4-a99b-6341945b7c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:22.446540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd30ce8-5496-46f4-a99b-6341945b7c77 admin admin] PATH_INFO: `/v3/projects/6a444937601d405f8aff6fe5eb290d57/users/ed0f815d986347189c3cbc1312b4b755/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:22.491057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd30ce8-5496-46f4-a99b-6341945b7c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a444937601d405f8aff6fe5eb290d57, user_id=ed0f815d986347189c3cbc1312b4b755, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:22.500369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd30ce8-5496-46f4-a99b-6341945b7c77 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:22.680457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 864/1726] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:22 2022] PUT /identity/v3/projects/6a444937601d405f8aff6fe5eb290d57/users/ed0f815d986347189c3cbc1312b4b755/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:22.685912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a47d-4458-4e2b-a85c-ebea2e98bfee None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:22.686422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a47d-4458-4e2b-a85c-ebea2e98bfee None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:22.686527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a47d-4458-4e2b-a85c-ebea2e98bfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:22.708764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b3c6a47d-4458-4e2b-a85c-ebea2e98bfee None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:22.710054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b3c6a47d-4458-4e2b-a85c-ebea2e98bfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:22.812080 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 863/1727] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:22.822063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd30ce8-5496-46f4-a99b-6341945b7c77 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:22.824266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6fd30ce8-5496-46f4-a99b-6341945b7c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:22.932792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd30ce8-5496-46f4-a99b-6341945b7c77 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:22.939831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3a811-f719-4c2f-b657-1ee09f4d53fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:22.941244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3a811-f719-4c2f-b657-1ee09f4d53fa service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:22.942927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3a811-f719-4c2f-b657-1ee09f4d53fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:22.943450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3a811-f719-4c2f-b657-1ee09f4d53fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:22.982316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3a811-f719-4c2f-b657-1ee09f4d53fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:22.985374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3a811-f719-4c2f-b657-1ee09f4d53fa service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:23.016475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 865/1728] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:26.492734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36007d-1d32-413b-8fea-6359fd38d2cc None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:26.493690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36007d-1d32-413b-8fea-6359fd38d2cc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:26.494110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36007d-1d32-413b-8fea-6359fd38d2cc None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:26.496886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 864/1729] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:09:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:09:26.525008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbd166-453d-4bc5-aee7-39fec6390889 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:26.525560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbd166-453d-4bc5-aee7-39fec6390889 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:26.526003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbd166-453d-4bc5-aee7-39fec6390889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:26.643485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-66bbd166-453d-4bc5-aee7-39fec6390889 None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:26.648023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-66bbd166-453d-4bc5-aee7-39fec6390889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:26.827647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:26.849580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 866/1730] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:09:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:27.100888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36007d-1d32-413b-8fea-6359fd38d2cc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:27.105349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fd36007d-1d32-413b-8fea-6359fd38d2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:27.162439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36007d-1d32-413b-8fea-6359fd38d2cc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:27.174826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:27.175596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:27.175986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:27.176355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] PATH_INFO: `/v3/users/37b4e16ac3f74fde82ebe5ed27662767` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:27.178285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=37b4e16ac3f74fde82ebe5ed27662767)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:27.186478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:27.196414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8da5cd-ec45-4f72-8922-6fce4a2cf0c3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:27.197003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8da5cd-ec45-4f72-8922-6fce4a2cf0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:27.197451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8da5cd-ec45-4f72-8922-6fce4a2cf0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:27.210643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:27.277186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6a8da5cd-ec45-4f72-8922-6fce4a2cf0c3 None None] MFA Rules not processed for user `ed0f815d986347189c3cbc1312b4b755`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:27.279391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6a8da5cd-ec45-4f72-8922-6fce4a2cf0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:27.430856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 867/1731] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:27 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:09:27.435119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8da5cd-ec45-4f72-8922-6fce4a2cf0c3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:27.467792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8da5cd-ec45-4f72-8922-6fce4a2cf0c3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:27.491577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26a6c9-d637-4d07-8f6b-3f0e77c1083f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:27.492590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26a6c9-d637-4d07-8f6b-3f0e77c1083f admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:27.492951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26a6c9-d637-4d07-8f6b-3f0e77c1083f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:27.493352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26a6c9-d637-4d07-8f6b-3f0e77c1083f admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:27.494177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26a6c9-d637-4d07-8f6b-3f0e77c1083f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:27.497312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26a6c9-d637-4d07-8f6b-3f0e77c1083f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:27.499552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37b4e16ac3f74fde82ebe5ed27662767'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:27.626778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37b4e16ac3f74fde82ebe5ed27662767'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:27.639726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37b4e16ac3f74fde82ebe5ed27662767'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:27.641641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37b4e16ac3f74fde82ebe5ed27662767'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:27.643297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37b4e16ac3f74fde82ebe5ed27662767'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:27.664581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 868/1732] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:27 2022] POST /identity/v3/projects => generated 403 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:27.665606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26a6c9-d637-4d07-8f6b-3f0e77c1083f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:27.676404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1e26a6c9-d637-4d07-8f6b-3f0e77c1083f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:27.719277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 865/1733] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:27 2022] DELETE /identity/v3/users/37b4e16ac3f74fde82ebe5ed27662767 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:27.719277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:27.729044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:27.919758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26a6c9-d637-4d07-8f6b-3f0e77c1083f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:27.938174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee76e0-d33e-4559-8034-91e4b9eaeddd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:27.952394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07099da-cfca-4db2-a375-98848b6883db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:27.953096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07099da-cfca-4db2-a375-98848b6883db admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:27.953406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07099da-cfca-4db2-a375-98848b6883db admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:27.953638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07099da-cfca-4db2-a375-98848b6883db admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:27.956277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07099da-cfca-4db2-a375-98848b6883db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:27.961800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07099da-cfca-4db2-a375-98848b6883db admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:28.029483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:28.039916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d57c9f-e79b-4345-9d3a-0100dd3304ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:28.040589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d57c9f-e79b-4345-9d3a-0100dd3304ff service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:28.040879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d57c9f-e79b-4345-9d3a-0100dd3304ff service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:28.041138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d57c9f-e79b-4345-9d3a-0100dd3304ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:28.043302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-21d57c9f-e79b-4345-9d3a-0100dd3304ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:28.063365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d57c9f-e79b-4345-9d3a-0100dd3304ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:28.065057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d57c9f-e79b-4345-9d3a-0100dd3304ff service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:28.085343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 866/1734] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:27 2022] POST /identity/v3/users => generated 308 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:28.087595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77545bde-c566-4acb-9350-be2aba66eb70 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:28.087887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77545bde-c566-4acb-9350-be2aba66eb70 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:28.091581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77545bde-c566-4acb-9350-be2aba66eb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:28.132718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 869/1735] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:28.133537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d57c9f-e79b-4345-9d3a-0100dd3304ff service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:28.183618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-77545bde-c566-4acb-9350-be2aba66eb70 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:28.185658 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d57c9f-e79b-4345-9d3a-0100dd3304ff service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:28.189680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-77545bde-c566-4acb-9350-be2aba66eb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:28.204556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f214a-48be-4811-9a7e-a87265ebbca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:28.204556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f214a-48be-4811-9a7e-a87265ebbca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:28.204556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f214a-48be-4811-9a7e-a87265ebbca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:28.204556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f214a-48be-4811-9a7e-a87265ebbca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:28.208604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f214a-48be-4811-9a7e-a87265ebbca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:28.210483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f214a-48be-4811-9a7e-a87265ebbca1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:28.233456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 867/1736] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:28.256980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 870/1737] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:28.259677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77545bde-c566-4acb-9350-be2aba66eb70 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:28.283121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77545bde-c566-4acb-9350-be2aba66eb70 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:28.312982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5cb62-4e42-4ec1-84cd-52945ffd8404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:28.314256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5cb62-4e42-4ec1-84cd-52945ffd8404 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:28.314789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5cb62-4e42-4ec1-84cd-52945ffd8404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:28.315854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5cb62-4e42-4ec1-84cd-52945ffd8404 admin admin] PATH_INFO: `/v3/projects/ccb117fb1fe945329e7420e9903f2096/users/4ee9242816aa48ed853596001bfe74fd/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:28.342594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5cb62-4e42-4ec1-84cd-52945ffd8404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb117fb1fe945329e7420e9903f2096, user_id=4ee9242816aa48ed853596001bfe74fd, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:28.347023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5cb62-4e42-4ec1-84cd-52945ffd8404 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:28.348825 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f214a-48be-4811-9a7e-a87265ebbca1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:28.350575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-904f214a-48be-4811-9a7e-a87265ebbca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:28.443240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 868/1738] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:28 2022] PUT /identity/v3/projects/ccb117fb1fe945329e7420e9903f2096/users/4ee9242816aa48ed853596001bfe74fd/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:28.444547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5cb62-4e42-4ec1-84cd-52945ffd8404 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:28.482975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f214a-48be-4811-9a7e-a87265ebbca1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:28.608325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:28.608942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb512a32-8c1a-4787-bf4a-77aae6d5d231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:28.609682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb512a32-8c1a-4787-bf4a-77aae6d5d231 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:28.609682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb512a32-8c1a-4787-bf4a-77aae6d5d231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:28.611171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb512a32-8c1a-4787-bf4a-77aae6d5d231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:28.659209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5cb62-4e42-4ec1-84cd-52945ffd8404 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:28.665986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07ae27-6c73-4341-b9d8-ecdf025735a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:28.667661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07ae27-6c73-4341-b9d8-ecdf025735a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:28.668258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07ae27-6c73-4341-b9d8-ecdf025735a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:28.668692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07ae27-6c73-4341-b9d8-ecdf025735a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:28.669489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07ae27-6c73-4341-b9d8-ecdf025735a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:28.671599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07ae27-6c73-4341-b9d8-ecdf025735a3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:28.690811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 869/1739] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:28.692422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07ae27-6c73-4341-b9d8-ecdf025735a3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:28.707999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07ae27-6c73-4341-b9d8-ecdf025735a3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:28.715584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8eebf-f552-4692-b0cc-6becfc3a16b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:28.718160 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8eebf-f552-4692-b0cc-6becfc3a16b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:28.718649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8eebf-f552-4692-b0cc-6becfc3a16b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:28.719455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8eebf-f552-4692-b0cc-6becfc3a16b7 admin admin] PATH_INFO: `/v3/projects/ccb117fb1fe945329e7420e9903f2096/users/4ee9242816aa48ed853596001bfe74fd/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:28.723955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8eebf-f552-4692-b0cc-6becfc3a16b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb117fb1fe945329e7420e9903f2096, user_id=4ee9242816aa48ed853596001bfe74fd, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:28.727874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8eebf-f552-4692-b0cc-6becfc3a16b7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:28.759589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 870/1740] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:28 2022] PUT /identity/v3/projects/ccb117fb1fe945329e7420e9903f2096/users/4ee9242816aa48ed853596001bfe74fd/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:28.781473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb512a32-8c1a-4787-bf4a-77aae6d5d231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:28.793857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb512a32-8c1a-4787-bf4a-77aae6d5d231 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:28.844833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 871/1741] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:09:29.873359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8eebf-f552-4692-b0cc-6becfc3a16b7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:29.875255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-61b8eebf-f552-4692-b0cc-6becfc3a16b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:30.079391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8eebf-f552-4692-b0cc-6becfc3a16b7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:30.084677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:30.101556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:30.102012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:30.102516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] PATH_INFO: `/v3/projects/5e4bf79da0f345a5a8a5a7565a08890e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:30.107512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4bf79da0f345a5a8a5a7565a08890e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:30.110649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:30.287602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4bf79da0f345a5a8a5a7565a08890e', 'request_id': 'req-fa24f738-6767-475d-bcb1-4efdb91ef4b1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.301846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4bf79da0f345a5a8a5a7565a08890e', 'request_id': 'req-fa24f738-6767-475d-bcb1-4efdb91ef4b1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.397856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4bf79da0f345a5a8a5a7565a08890e', 'request_id': 'req-fa24f738-6767-475d-bcb1-4efdb91ef4b1'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.420653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] The token cache is being invalidate because project 5e4bf79da0f345a5a8a5a7565a08890e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:30.421365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.422470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.428883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 871/1742] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:29 2022] DELETE /identity/v3/projects/5e4bf79da0f345a5a8a5a7565a08890e => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:30.430121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb512a32-8c1a-4787-bf4a-77aae6d5d231 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:30.440089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb512a32-8c1a-4787-bf4a-77aae6d5d231 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:30.489162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:30.507392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:30.508369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:30.508772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:30.509143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] PATH_INFO: `/v3/users/5f85ee2020914f988fc8fdc22680d911` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:30.511178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f85ee2020914f988fc8fdc22680d911)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:30.513291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:30.764225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f85ee2020914f988fc8fdc22680d911'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.847255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f85ee2020914f988fc8fdc22680d911'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.869332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f85ee2020914f988fc8fdc22680d911'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.870486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f85ee2020914f988fc8fdc22680d911'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.871359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-bf11c3a9-52d1-4429-a2cf-ef628a698909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f85ee2020914f988fc8fdc22680d911'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:30.896645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 872/1743] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:30 2022] DELETE /identity/v3/users/5f85ee2020914f988fc8fdc22680d911 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:09:31.598313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:31.599972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:31.615987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24f738-6767-475d-bcb1-4efdb91ef4b1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:31.696727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:31.702870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:31.703664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:31.704019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:31.704440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] PATH_INFO: `/v3/projects/c3f682e9c9944f23a5c3b54fa69e6599` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:31.705718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f682e9c9944f23a5c3b54fa69e6599)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:31.707794 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:31.780428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cbb9c-82d5-4e60-9f5d-2135f2b4eae2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:31.780993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cbb9c-82d5-4e60-9f5d-2135f2b4eae2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:31.781079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cbb9c-82d5-4e60-9f5d-2135f2b4eae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:31.834005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f682e9c9944f23a5c3b54fa69e6599', 'request_id': 'req-09eb80e1-27ae-434f-9be4-cac0cce12b62'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:31.836265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-198cbb9c-82d5-4e60-9f5d-2135f2b4eae2 None None] MFA Rules not processed for user `4ee9242816aa48ed853596001bfe74fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:31.838491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-198cbb9c-82d5-4e60-9f5d-2135f2b4eae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:31.843594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f682e9c9944f23a5c3b54fa69e6599', 'request_id': 'req-09eb80e1-27ae-434f-9be4-cac0cce12b62'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:31.922772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f682e9c9944f23a5c3b54fa69e6599', 'request_id': 'req-09eb80e1-27ae-434f-9be4-cac0cce12b62'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:31.945911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] The token cache is being invalidate because project c3f682e9c9944f23a5c3b54fa69e6599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:31.945911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:31.945911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09eb80e1-27ae-434f-9be4-cac0cce12b62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:31.945911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 872/1744] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:31 2022] DELETE /identity/v3/projects/c3f682e9c9944f23a5c3b54fa69e6599 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:31.948606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b364b-523d-4786-a089-79bf29b248af None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:31.948952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b364b-523d-4786-a089-79bf29b248af None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:31.950245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b364b-523d-4786-a089-79bf29b248af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:31.956542 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 873/1745] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:31 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:09:31.978288 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-565b364b-523d-4786-a089-79bf29b248af None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:31.979552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-565b364b-523d-4786-a089-79bf29b248af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:32.061041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 873/1746] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:32.068243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dbbb1-6ed3-41b3-b5b1-36e185824428 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.068728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dbbb1-6ed3-41b3-b5b1-36e185824428 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.069209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dbbb1-6ed3-41b3-b5b1-36e185824428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.090954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ce6dbbb1-6ed3-41b3-b5b1-36e185824428 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:32.092634 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ce6dbbb1-6ed3-41b3-b5b1-36e185824428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:32.101298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 874/1747] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:32.105589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b364b-523d-4786-a089-79bf29b248af None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:32.115791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b364b-523d-4786-a089-79bf29b248af None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:32.120455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4bd90-ea15-4d8f-9e91-6ef9479edba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:32.121282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4bd90-ea15-4d8f-9e91-6ef9479edba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.121479 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4bd90-ea15-4d8f-9e91-6ef9479edba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.121752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4bd90-ea15-4d8f-9e91-6ef9479edba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.122431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4bd90-ea15-4d8f-9e91-6ef9479edba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:09:32.122729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4bd90-ea15-4d8f-9e91-6ef9479edba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:32.124327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4bd90-ea15-4d8f-9e91-6ef9479edba7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:32.136647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 874/1748] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:09:32.141083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dbbb1-6ed3-41b3-b5b1-36e185824428 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:32.150336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dbbb1-6ed3-41b3-b5b1-36e185824428 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:32.155456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eab95e-a1e5-46ee-a181-2d934a7966ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:32.160959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab95e-a1e5-46ee-a181-2d934a7966ab admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.160959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab95e-a1e5-46ee-a181-2d934a7966ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.160959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab95e-a1e5-46ee-a181-2d934a7966ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.160959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eab95e-a1e5-46ee-a181-2d934a7966ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:32.160959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eab95e-a1e5-46ee-a181-2d934a7966ab admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:32.309445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 875/1749] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:32 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:09:32.309773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4bd90-ea15-4d8f-9e91-6ef9479edba7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:32.321386 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4bd90-ea15-4d8f-9e91-6ef9479edba7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:32.395909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe72d8-f443-4e3c-8597-98e226bf7e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:32.397470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe72d8-f443-4e3c-8597-98e226bf7e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.402316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe72d8-f443-4e3c-8597-98e226bf7e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.402316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe72d8-f443-4e3c-8597-98e226bf7e8e admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.402316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe72d8-f443-4e3c-8597-98e226bf7e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:32.402316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe72d8-f443-4e3c-8597-98e226bf7e8e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:32.483532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 875/1750] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:32 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:09:32.487529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eab95e-a1e5-46ee-a181-2d934a7966ab admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:32.498731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eab95e-a1e5-46ee-a181-2d934a7966ab admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:32.503507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30150007-0d1b-40ca-aaa8-7363b69d23bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:32.504144 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30150007-0d1b-40ca-aaa8-7363b69d23bf admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.504424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30150007-0d1b-40ca-aaa8-7363b69d23bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.504723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30150007-0d1b-40ca-aaa8-7363b69d23bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.505395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30150007-0d1b-40ca-aaa8-7363b69d23bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:32.507113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30150007-0d1b-40ca-aaa8-7363b69d23bf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:32.521180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 876/1751] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:32.522369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe72d8-f443-4e3c-8597-98e226bf7e8e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:32.532605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe72d8-f443-4e3c-8597-98e226bf7e8e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:32.538977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102b372-4270-44e5-8b8f-f6c7c70f6675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:32.539817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102b372-4270-44e5-8b8f-f6c7c70f6675 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.540199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102b372-4270-44e5-8b8f-f6c7c70f6675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.540725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102b372-4270-44e5-8b8f-f6c7c70f6675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.541379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102b372-4270-44e5-8b8f-f6c7c70f6675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:32.546507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102b372-4270-44e5-8b8f-f6c7c70f6675 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:32.553723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 876/1752] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:32.558469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30150007-0d1b-40ca-aaa8-7363b69d23bf admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:32.563101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:32.568913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30150007-0d1b-40ca-aaa8-7363b69d23bf admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:32.574188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e5382-bf68-4e54-be87-6f95c164af05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:32.575112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e5382-bf68-4e54-be87-6f95c164af05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.575425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e5382-bf68-4e54-be87-6f95c164af05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.575703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e5382-bf68-4e54-be87-6f95c164af05 admin admin] PATH_INFO: `/v3/projects/f45909ec6958425f8ff329a028e95ba5/users/bbefd55d7a784386ae325dcab3eeb14f/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.577934 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:32.591677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e5382-bf68-4e54-be87-6f95c164af05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f45909ec6958425f8ff329a028e95ba5, user_id=bbefd55d7a784386ae325dcab3eeb14f, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:32.594364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e5382-bf68-4e54-be87-6f95c164af05 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:32.659458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 877/1753] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:32 2022] PUT /identity/v3/projects/f45909ec6958425f8ff329a028e95ba5/users/bbefd55d7a784386ae325dcab3eeb14f/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:32.670850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b085b-5df6-4771-9fad-fc1f25b2bfed None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.672828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b085b-5df6-4771-9fad-fc1f25b2bfed None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.672828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b085b-5df6-4771-9fad-fc1f25b2bfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.699188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d66b085b-5df6-4771-9fad-fc1f25b2bfed None None] MFA Rules not processed for user `bbefd55d7a784386ae325dcab3eeb14f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:32.700675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d66b085b-5df6-4771-9fad-fc1f25b2bfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:32.771227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 877/1754] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:32.779933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e5382-bf68-4e54-be87-6f95c164af05 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:32.802007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658a442-c99b-4349-aa2a-c6be22f6cc60 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.802553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658a442-c99b-4349-aa2a-c6be22f6cc60 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.803107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658a442-c99b-4349-aa2a-c6be22f6cc60 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.805076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 878/1755] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:09:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:09:32.831052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93b550-d5a4-4997-9ae0-fcc3d799fe50 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.831280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93b550-d5a4-4997-9ae0-fcc3d799fe50 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.831549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93b550-d5a4-4997-9ae0-fcc3d799fe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.845534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e5382-bf68-4e54-be87-6f95c164af05 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:32.853468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ae93b550-d5a4-4997-9ae0-fcc3d799fe50 None None] MFA Rules not processed for user `b81bdfeedcba4b0c93f13d14318b2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:32.855007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ae93b550-d5a4-4997-9ae0-fcc3d799fe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:32.855199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18160548-4fe3-41fa-9f92-96d0e8bd1c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:32.855827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18160548-4fe3-41fa-9f92-96d0e8bd1c2e admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:32.857060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18160548-4fe3-41fa-9f92-96d0e8bd1c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:32.857452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18160548-4fe3-41fa-9f92-96d0e8bd1c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:32.858270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18160548-4fe3-41fa-9f92-96d0e8bd1c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:32.860451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18160548-4fe3-41fa-9f92-96d0e8bd1c2e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:32.926020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 879/1756] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:09:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:32.946871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93b550-d5a4-4997-9ae0-fcc3d799fe50 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:32.949156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ae93b550-d5a4-4997-9ae0-fcc3d799fe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:32.950601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 878/1757] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:32 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:09:32.954175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18160548-4fe3-41fa-9f92-96d0e8bd1c2e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:32.969227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93b550-d5a4-4997-9ae0-fcc3d799fe50 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:33.042608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18160548-4fe3-41fa-9f92-96d0e8bd1c2e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:33.047512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cc580-a1cd-4b61-b5e6-f85b30b8e6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:33.048283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cc580-a1cd-4b61-b5e6-f85b30b8e6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:33.048653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cc580-a1cd-4b61-b5e6-f85b30b8e6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:33.048829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b07f9-9db6-46c4-8ad3-43f2fbb176b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:33.049121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cc580-a1cd-4b61-b5e6-f85b30b8e6b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:33.049680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b07f9-9db6-46c4-8ad3-43f2fbb176b2 service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:33.049870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2cc580-a1cd-4b61-b5e6-f85b30b8e6b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:33.050070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b07f9-9db6-46c4-8ad3-43f2fbb176b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:33.050354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b07f9-9db6-46c4-8ad3-43f2fbb176b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:33.052845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2cc580-a1cd-4b61-b5e6-f85b30b8e6b4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:33.061207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b07f9-9db6-46c4-8ad3-43f2fbb176b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:33.062866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b07f9-9db6-46c4-8ad3-43f2fbb176b2 service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:33.074052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 880/1758] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:09:33.166548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 879/1759] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:32 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:09:33.167747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b07f9-9db6-46c4-8ad3-43f2fbb176b2 service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:33.178280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b07f9-9db6-46c4-8ad3-43f2fbb176b2 service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:33.184347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cdc79-89da-4b7d-93b5-408b58b7135d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:33.185183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cdc79-89da-4b7d-93b5-408b58b7135d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:33.185357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cdc79-89da-4b7d-93b5-408b58b7135d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:33.185719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cdc79-89da-4b7d-93b5-408b58b7135d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:33.186421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cdc79-89da-4b7d-93b5-408b58b7135d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:33.188101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cdc79-89da-4b7d-93b5-408b58b7135d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:33.201682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 881/1760] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:33.206438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cc580-a1cd-4b61-b5e6-f85b30b8e6b4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:33.215308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:33.219223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cc580-a1cd-4b61-b5e6-f85b30b8e6b4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:33.236682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa173b-9b3f-44fe-b3d6-e0632ed69648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:33.236682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa173b-9b3f-44fe-b3d6-e0632ed69648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:33.236682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa173b-9b3f-44fe-b3d6-e0632ed69648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:33.236682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa173b-9b3f-44fe-b3d6-e0632ed69648 admin admin] PATH_INFO: `/v3/projects/13136c5b088c4a1994408eb1781d4be4/users/e0b7ff6d253249a892837e03dade4aba/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:33.242599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa173b-9b3f-44fe-b3d6-e0632ed69648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13136c5b088c4a1994408eb1781d4be4, user_id=e0b7ff6d253249a892837e03dade4aba, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:33.245462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa173b-9b3f-44fe-b3d6-e0632ed69648 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:33.352641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 880/1761] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:33 2022] PUT /identity/v3/projects/13136c5b088c4a1994408eb1781d4be4/users/e0b7ff6d253249a892837e03dade4aba/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:33.357809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cdc79-89da-4b7d-93b5-408b58b7135d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:33.370107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cdc79-89da-4b7d-93b5-408b58b7135d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:33.432680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbaf5bd-a942-4432-8a80-e412315b982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:33.433294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaf5bd-a942-4432-8a80-e412315b982c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:33.433573 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaf5bd-a942-4432-8a80-e412315b982c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:33.433851 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaf5bd-a942-4432-8a80-e412315b982c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:33.434567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbaf5bd-a942-4432-8a80-e412315b982c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:33.436452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbaf5bd-a942-4432-8a80-e412315b982c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:33.457927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 882/1762] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:33.462299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa173b-9b3f-44fe-b3d6-e0632ed69648 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:33.479477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa173b-9b3f-44fe-b3d6-e0632ed69648 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:33.486914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350595e4-fe93-4056-ac8d-26bcd2266bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:33.487344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350595e4-fe93-4056-ac8d-26bcd2266bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:33.487721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350595e4-fe93-4056-ac8d-26bcd2266bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:33.488087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350595e4-fe93-4056-ac8d-26bcd2266bcd admin admin] PATH_INFO: `/v3/projects/13136c5b088c4a1994408eb1781d4be4/users/e0b7ff6d253249a892837e03dade4aba/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:33.490276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350595e4-fe93-4056-ac8d-26bcd2266bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=13136c5b088c4a1994408eb1781d4be4, user_id=e0b7ff6d253249a892837e03dade4aba, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:33.493110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350595e4-fe93-4056-ac8d-26bcd2266bcd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:33.506553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 881/1763] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:33 2022] PUT /identity/v3/projects/13136c5b088c4a1994408eb1781d4be4/users/e0b7ff6d253249a892837e03dade4aba/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:33.517496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c369ce8-d554-4f86-8bb7-3209b06367a0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:33.517804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c369ce8-d554-4f86-8bb7-3209b06367a0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:33.518466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c369ce8-d554-4f86-8bb7-3209b06367a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:33.550543 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3c369ce8-d554-4f86-8bb7-3209b06367a0 None None] MFA Rules not processed for user `e0b7ff6d253249a892837e03dade4aba`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:33.551784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3c369ce8-d554-4f86-8bb7-3209b06367a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:33.622710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 883/1764] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:33.633889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350595e4-fe93-4056-ac8d-26bcd2266bcd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:33.636216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-350595e4-fe93-4056-ac8d-26bcd2266bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:33.660520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:33.699367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350595e4-fe93-4056-ac8d-26bcd2266bcd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:33.703322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a5a10-088c-466a-9945-8a16419f1684 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:33.704338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a5a10-088c-466a-9945-8a16419f1684 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:33.705004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a5a10-088c-466a-9945-8a16419f1684 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:33.708676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a5a10-088c-466a-9945-8a16419f1684 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:33.716359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a5a10-088c-466a-9945-8a16419f1684 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:33.717911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a5a10-088c-466a-9945-8a16419f1684 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:33.727837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 882/1765] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:34.020193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c369ce8-d554-4f86-8bb7-3209b06367a0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:34.021895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3c369ce8-d554-4f86-8bb7-3209b06367a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:34.040830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09eea2-abba-4112-91c7-af9087998908 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:34.041379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09eea2-abba-4112-91c7-af9087998908 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:34.041486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09eea2-abba-4112-91c7-af9087998908 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:34.042790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 883/1766] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:09:34.044305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c369ce8-d554-4f86-8bb7-3209b06367a0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:34.047982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf284c2b-fc34-4b5c-8d9a-5ea48e7348f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:34.048629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf284c2b-fc34-4b5c-8d9a-5ea48e7348f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:34.048908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf284c2b-fc34-4b5c-8d9a-5ea48e7348f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:34.049174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf284c2b-fc34-4b5c-8d9a-5ea48e7348f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:34.074295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadec95-b2cf-4afd-9006-f67f1abad54d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:34.074833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadec95-b2cf-4afd-9006-f67f1abad54d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:34.075105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadec95-b2cf-4afd-9006-f67f1abad54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:34.097054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5eadec95-b2cf-4afd-9006-f67f1abad54d None None] MFA Rules not processed for user `b81bdfeedcba4b0c93f13d14318b2151`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:34.098271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5eadec95-b2cf-4afd-9006-f67f1abad54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:34.112545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf284c2b-fc34-4b5c-8d9a-5ea48e7348f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:34.113991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf284c2b-fc34-4b5c-8d9a-5ea48e7348f8 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:34.123909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 884/1767] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:34.177643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 884/1768] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:09:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:09:34.190096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf284c2b-fc34-4b5c-8d9a-5ea48e7348f8 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:34.213731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadec95-b2cf-4afd-9006-f67f1abad54d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:34.218416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5eadec95-b2cf-4afd-9006-f67f1abad54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:34.224138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf284c2b-fc34-4b5c-8d9a-5ea48e7348f8 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:34.234078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07cedb-d929-448c-9b63-c3e0166e3d43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:34.235044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07cedb-d929-448c-9b63-c3e0166e3d43 service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:34.235570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07cedb-d929-448c-9b63-c3e0166e3d43 service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:34.236038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07cedb-d929-448c-9b63-c3e0166e3d43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:34.259770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07cedb-d929-448c-9b63-c3e0166e3d43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:34.268107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07cedb-d929-448c-9b63-c3e0166e3d43 service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:34.302505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 885/1769] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:34.323926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadec95-b2cf-4afd-9006-f67f1abad54d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:34.330145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:34.330859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:34.331198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:34.331546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] PATH_INFO: `/v3/users/ed0f815d986347189c3cbc1312b4b755` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:34.332889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0f815d986347189c3cbc1312b4b755)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:34.335076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:34.634062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0f815d986347189c3cbc1312b4b755'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:34.746209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0f815d986347189c3cbc1312b4b755'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:34.756153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:34.756153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0f815d986347189c3cbc1312b4b755'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:34.756933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0f815d986347189c3cbc1312b4b755'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:34.757949 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0f815d986347189c3cbc1312b4b755'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:34.775837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 885/1770] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:34 2022] DELETE /identity/v3/users/ed0f815d986347189c3cbc1312b4b755 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:35.098613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07cedb-d929-448c-9b63-c3e0166e3d43 service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:35.105483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:35.110334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07cedb-d929-448c-9b63-c3e0166e3d43 service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:35.189865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:35.190456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:35.190730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:35.190994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] PATH_INFO: `/v3/projects/6a444937601d405f8aff6fe5eb290d57` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:35.192323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a444937601d405f8aff6fe5eb290d57)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:35.194047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:35.288729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a444937601d405f8aff6fe5eb290d57', 'request_id': 'req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.295171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a444937601d405f8aff6fe5eb290d57', 'request_id': 'req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.356615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a444937601d405f8aff6fe5eb290d57', 'request_id': 'req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.371525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] The token cache is being invalidate because project 6a444937601d405f8aff6fe5eb290d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:35.372025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.373059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.375352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 886/1771] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:35 2022] DELETE /identity/v3/projects/6a444937601d405f8aff6fe5eb290d57 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:35.380429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:35.381869 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:35.395155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d26fa-6804-4240-ba4c-e90d2056d5d5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:35.454826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:35.457689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:35.457689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:35.457689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] PATH_INFO: `/v3/users/4ee9242816aa48ed853596001bfe74fd` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:35.458695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee9242816aa48ed853596001bfe74fd)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:35.461953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:35.671117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee9242816aa48ed853596001bfe74fd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.763601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee9242816aa48ed853596001bfe74fd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.773443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee9242816aa48ed853596001bfe74fd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.776457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee9242816aa48ed853596001bfe74fd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.777245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-024bf376-fa03-4b53-bbd6-2ea91246b6e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee9242816aa48ed853596001bfe74fd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:35.815650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 886/1772] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:35 2022] DELETE /identity/v3/users/4ee9242816aa48ed853596001bfe74fd => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:09:36.078991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:36.080574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:36.104418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f47129-f52c-4a00-8b6c-ea6dfae59f7f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:36.171345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:36.173358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:36.173358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:36.173358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] PATH_INFO: `/v3/projects/ccb117fb1fe945329e7420e9903f2096` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:36.173798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb117fb1fe945329e7420e9903f2096)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:36.175710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:36.304016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:36.314892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb117fb1fe945329e7420e9903f2096', 'request_id': 'req-293747be-56ec-4f0b-9fd9-a42814648dc6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:36.320408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb117fb1fe945329e7420e9903f2096', 'request_id': 'req-293747be-56ec-4f0b-9fd9-a42814648dc6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:36.413960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb117fb1fe945329e7420e9903f2096', 'request_id': 'req-293747be-56ec-4f0b-9fd9-a42814648dc6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:36.425336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] The token cache is being invalidate because project ccb117fb1fe945329e7420e9903f2096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:36.425837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:36.426635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-293747be-56ec-4f0b-9fd9-a42814648dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:36.428446 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 887/1773] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:36 2022] DELETE /identity/v3/projects/ccb117fb1fe945329e7420e9903f2096 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:36.433364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c543a-f320-451e-9d59-5bde759c1e32 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:36.438424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c543a-f320-451e-9d59-5bde759c1e32 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:36.438424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c543a-f320-451e-9d59-5bde759c1e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:36.460632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cc5c543a-f320-451e-9d59-5bde759c1e32 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:36.462529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cc5c543a-f320-451e-9d59-5bde759c1e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:36.531479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:36.540465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 887/1774] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:36.548077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8dcab-297b-4e49-abb9-50c7e272608b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:36.548356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8dcab-297b-4e49-abb9-50c7e272608b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:36.548657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8dcab-297b-4e49-abb9-50c7e272608b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:36.567930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-08f8dcab-297b-4e49-abb9-50c7e272608b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:36.569165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-08f8dcab-297b-4e49-abb9-50c7e272608b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:36.578584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 888/1775] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:36.582791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c543a-f320-451e-9d59-5bde759c1e32 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:36.593114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c543a-f320-451e-9d59-5bde759c1e32 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:36.598193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecd156-8c8e-41fa-ac71-575795513490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:36.598767 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecd156-8c8e-41fa-ac71-575795513490 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:36.599037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecd156-8c8e-41fa-ac71-575795513490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:36.599296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecd156-8c8e-41fa-ac71-575795513490 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:36.599930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ecd156-8c8e-41fa-ac71-575795513490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:09:36.600220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ecd156-8c8e-41fa-ac71-575795513490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:36.601878 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ecd156-8c8e-41fa-ac71-575795513490 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:36.613238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 888/1776] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:09:36.618155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8dcab-297b-4e49-abb9-50c7e272608b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:36.627490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8dcab-297b-4e49-abb9-50c7e272608b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:36.632856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf4186-4de3-4a49-9d1d-0e4759cdd28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:36.633472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf4186-4de3-4a49-9d1d-0e4759cdd28b admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:36.633745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf4186-4de3-4a49-9d1d-0e4759cdd28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:36.634010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf4186-4de3-4a49-9d1d-0e4759cdd28b admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:36.635111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf4186-4de3-4a49-9d1d-0e4759cdd28b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:36.637114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf4186-4de3-4a49-9d1d-0e4759cdd28b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:37.620351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 889/1777] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:36 2022] POST /identity/v3/projects => generated 357 bytes in 1003 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:37.631510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecd156-8c8e-41fa-ac71-575795513490 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:37.635055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecd156-8c8e-41fa-ac71-575795513490 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:37.716158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fa4e7-823f-4745-b603-fff8658be6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:37.717073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fa4e7-823f-4745-b603-fff8658be6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:37.717412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fa4e7-823f-4745-b603-fff8658be6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:37.717745 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fa4e7-823f-4745-b603-fff8658be6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:37.718463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fa4e7-823f-4745-b603-fff8658be6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:37.722425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fa4e7-823f-4745-b603-fff8658be6c2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:37.883093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 889/1778] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:37 2022] POST /identity/v3/users => generated 279 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:09:37.883666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf4186-4de3-4a49-9d1d-0e4759cdd28b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:37.895414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf4186-4de3-4a49-9d1d-0e4759cdd28b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:37.905248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687f888-e58f-4112-8150-2cdb1b2140af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:37.905928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687f888-e58f-4112-8150-2cdb1b2140af admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:37.906772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687f888-e58f-4112-8150-2cdb1b2140af admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:37.907050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687f888-e58f-4112-8150-2cdb1b2140af admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:37.907756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687f888-e58f-4112-8150-2cdb1b2140af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:37.909613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687f888-e58f-4112-8150-2cdb1b2140af admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:37.923701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 890/1779] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:09:37.935178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fa4e7-823f-4745-b603-fff8658be6c2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:37.946758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fa4e7-823f-4745-b603-fff8658be6c2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:37.967523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a310813-955e-4d13-9e29-ab28541feacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:37.970082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a310813-955e-4d13-9e29-ab28541feacb admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:37.970460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a310813-955e-4d13-9e29-ab28541feacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:37.971646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a310813-955e-4d13-9e29-ab28541feacb admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:37.972738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a310813-955e-4d13-9e29-ab28541feacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:37.982438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a310813-955e-4d13-9e29-ab28541feacb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:38.001350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687f888-e58f-4112-8150-2cdb1b2140af admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:38.002532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 890/1780] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:38.014344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687f888-e58f-4112-8150-2cdb1b2140af admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:38.020188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b532f1-6884-40dd-9a8f-5e8a480eb157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:38.024752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b532f1-6884-40dd-9a8f-5e8a480eb157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:38.024752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b532f1-6884-40dd-9a8f-5e8a480eb157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:38.024752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b532f1-6884-40dd-9a8f-5e8a480eb157 admin admin] PATH_INFO: `/v3/projects/a395d0a9a4f44df09daaaeb1201cd9c9/users/ec887671b602492f817f229baf8a8803/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:38.037753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b532f1-6884-40dd-9a8f-5e8a480eb157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a395d0a9a4f44df09daaaeb1201cd9c9, user_id=ec887671b602492f817f229baf8a8803, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:38.040040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b532f1-6884-40dd-9a8f-5e8a480eb157 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:38.139365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 891/1781] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:38 2022] PUT /identity/v3/projects/a395d0a9a4f44df09daaaeb1201cd9c9/users/ec887671b602492f817f229baf8a8803/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:38.152035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2491717-468e-43b3-8db1-37df60721cd4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:38.152325 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2491717-468e-43b3-8db1-37df60721cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:38.153329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2491717-468e-43b3-8db1-37df60721cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:38.210943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a2491717-468e-43b3-8db1-37df60721cd4 None None] MFA Rules not processed for user `ec887671b602492f817f229baf8a8803`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:38.216634 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a2491717-468e-43b3-8db1-37df60721cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:38.392107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 891/1782] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:38.397013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b532f1-6884-40dd-9a8f-5e8a480eb157 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:38.496457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b532f1-6884-40dd-9a8f-5e8a480eb157 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:38.504208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f1b8-755e-4173-bf37-67553d30ec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:38.505083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f1b8-755e-4173-bf37-67553d30ec9a admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:38.505681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f1b8-755e-4173-bf37-67553d30ec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:38.506337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f1b8-755e-4173-bf37-67553d30ec9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:38.507658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946f1b8-755e-4173-bf37-67553d30ec9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:38.512080 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946f1b8-755e-4173-bf37-67553d30ec9a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:38.639061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 892/1783] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:38 2022] POST /identity/v3/projects => generated 355 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:38.643286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2491717-468e-43b3-8db1-37df60721cd4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:38.654909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2491717-468e-43b3-8db1-37df60721cd4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:38.720237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:38.729382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad5e9f-7e5d-4204-9e42-255705c790f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:38.729982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad5e9f-7e5d-4204-9e42-255705c790f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:38.730252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad5e9f-7e5d-4204-9e42-255705c790f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:38.730520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad5e9f-7e5d-4204-9e42-255705c790f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:38.731150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad5e9f-7e5d-4204-9e42-255705c790f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:38.733088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad5e9f-7e5d-4204-9e42-255705c790f7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:38.829643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 892/1784] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:38 2022] POST /identity/v3/users => generated 284 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:09:38.838428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f1b8-755e-4173-bf37-67553d30ec9a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:38.853147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f1b8-755e-4173-bf37-67553d30ec9a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:38.861361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e35d4-234e-44a4-a316-b52c9a8f85a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:38.862171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e35d4-234e-44a4-a316-b52c9a8f85a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:38.862440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e35d4-234e-44a4-a316-b52c9a8f85a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:38.863840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e35d4-234e-44a4-a316-b52c9a8f85a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:38.864871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e35d4-234e-44a4-a316-b52c9a8f85a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:38.866747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e35d4-234e-44a4-a316-b52c9a8f85a2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:38.880960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 893/1785] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:38.883950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad5e9f-7e5d-4204-9e42-255705c790f7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:38.894101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad5e9f-7e5d-4204-9e42-255705c790f7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:38.899396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5e9e9-a81c-4cb4-a3a4-1e58bc1bd021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:38.900093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5e9e9-a81c-4cb4-a3a4-1e58bc1bd021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:38.900411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5e9e9-a81c-4cb4-a3a4-1e58bc1bd021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:38.901856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5e9e9-a81c-4cb4-a3a4-1e58bc1bd021 admin admin] PATH_INFO: `/v3/projects/c72ef3caaa6449ee81ca7a08f7f55d6d/users/a3b928284cf64d2da7f0ae58f59f3939/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:38.920768 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5e9e9-a81c-4cb4-a3a4-1e58bc1bd021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72ef3caaa6449ee81ca7a08f7f55d6d, user_id=a3b928284cf64d2da7f0ae58f59f3939, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:38.922001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5e9e9-a81c-4cb4-a3a4-1e58bc1bd021 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:38.991562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 893/1786] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:38 2022] PUT /identity/v3/projects/c72ef3caaa6449ee81ca7a08f7f55d6d/users/a3b928284cf64d2da7f0ae58f59f3939/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:38.998476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e35d4-234e-44a4-a316-b52c9a8f85a2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:39.008419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e35d4-234e-44a4-a316-b52c9a8f85a2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:39.053749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:39.081855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415b94e-74f2-478d-ae75-af3dad88cd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:39.082471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415b94e-74f2-478d-ae75-af3dad88cd47 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:39.082750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415b94e-74f2-478d-ae75-af3dad88cd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:39.083430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415b94e-74f2-478d-ae75-af3dad88cd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:39.084539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415b94e-74f2-478d-ae75-af3dad88cd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:39.085279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415b94e-74f2-478d-ae75-af3dad88cd47 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:39.097077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 894/1787] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:39.102197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5e9e9-a81c-4cb4-a3a4-1e58bc1bd021 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:39.115727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5e9e9-a81c-4cb4-a3a4-1e58bc1bd021 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:39.121728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66865e53-b891-4709-b34e-784bf201bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:39.121948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66865e53-b891-4709-b34e-784bf201bf77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:39.122217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66865e53-b891-4709-b34e-784bf201bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:39.123639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66865e53-b891-4709-b34e-784bf201bf77 admin admin] PATH_INFO: `/v3/projects/c72ef3caaa6449ee81ca7a08f7f55d6d/users/a3b928284cf64d2da7f0ae58f59f3939/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:39.125545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66865e53-b891-4709-b34e-784bf201bf77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72ef3caaa6449ee81ca7a08f7f55d6d, user_id=a3b928284cf64d2da7f0ae58f59f3939, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:39.128823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66865e53-b891-4709-b34e-784bf201bf77 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:39.140854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 894/1788] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:39 2022] PUT /identity/v3/projects/c72ef3caaa6449ee81ca7a08f7f55d6d/users/a3b928284cf64d2da7f0ae58f59f3939/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:39.151523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec91dc-6b4b-4ead-92c3-acdcc3b74215 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:39.154795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec91dc-6b4b-4ead-92c3-acdcc3b74215 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:39.154795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec91dc-6b4b-4ead-92c3-acdcc3b74215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:39.189370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-5cec91dc-6b4b-4ead-92c3-acdcc3b74215 None None] MFA Rules not processed for user `a3b928284cf64d2da7f0ae58f59f3939`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:39.190688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5cec91dc-6b4b-4ead-92c3-acdcc3b74215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:39.288767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 895/1789] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:09:39.297374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66865e53-b891-4709-b34e-784bf201bf77 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:39.299659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-66865e53-b891-4709-b34e-784bf201bf77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:39.439488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66865e53-b891-4709-b34e-784bf201bf77 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:39.444553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d834d2-f7e3-497d-9b66-bc8e1be1f1ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:39.446344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d834d2-f7e3-497d-9b66-bc8e1be1f1ab service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:39.446344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d834d2-f7e3-497d-9b66-bc8e1be1f1ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:39.446344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d834d2-f7e3-497d-9b66-bc8e1be1f1ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:39.458662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855eb08-02bf-4861-84c9-0a36f10e275c None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:39.459283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855eb08-02bf-4861-84c9-0a36f10e275c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:39.459390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d834d2-f7e3-497d-9b66-bc8e1be1f1ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:39.459876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855eb08-02bf-4861-84c9-0a36f10e275c None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:39.461471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 896/1790] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:09:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 18:09:39.462648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d834d2-f7e3-497d-9b66-bc8e1be1f1ab service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:39.469798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8a621-cfe2-4490-ae90-c732337be540 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:39.471133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8a621-cfe2-4490-ae90-c732337be540 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:39.471606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8a621-cfe2-4490-ae90-c732337be540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:39.479070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 895/1791] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:39 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:39.497917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-5eb8a621-cfe2-4490-ae90-c732337be540 None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:39.499138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5eb8a621-cfe2-4490-ae90-c732337be540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:39.596698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 897/1792] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:09:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:39.711978 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d834d2-f7e3-497d-9b66-bc8e1be1f1ab service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:39.724700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d834d2-f7e3-497d-9b66-bc8e1be1f1ab service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:39.731387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d00fc8-a08e-4dd3-8169-a550c718a6c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:39.733041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d00fc8-a08e-4dd3-8169-a550c718a6c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:39.733668 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d00fc8-a08e-4dd3-8169-a550c718a6c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:39.739321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d00fc8-a08e-4dd3-8169-a550c718a6c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:39.831043 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d00fc8-a08e-4dd3-8169-a550c718a6c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:39.832614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d00fc8-a08e-4dd3-8169-a550c718a6c3 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:39.846090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 896/1793] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:39 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:39.913843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8a621-cfe2-4490-ae90-c732337be540 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:39.916557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5eb8a621-cfe2-4490-ae90-c732337be540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:39.930779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8a621-cfe2-4490-ae90-c732337be540 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:39.936410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b31c64-99e8-4d0e-ba23-ac72d7dec98b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:39.941484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b31c64-99e8-4d0e-ba23-ac72d7dec98b service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:39.941484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b31c64-99e8-4d0e-ba23-ac72d7dec98b service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:39.941484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b31c64-99e8-4d0e-ba23-ac72d7dec98b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:39.947201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b31c64-99e8-4d0e-ba23-ac72d7dec98b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:39.948841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b31c64-99e8-4d0e-ba23-ac72d7dec98b service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:39.961701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 898/1794] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:39.993320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d00fc8-a08e-4dd3-8169-a550c718a6c3 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:39.995554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-86d00fc8-a08e-4dd3-8169-a550c718a6c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:40.079535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:40.109262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d00fc8-a08e-4dd3-8169-a550c718a6c3 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:40.116894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:40.118052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:40.118461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:40.118734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] PATH_INFO: `/v3/users/bbefd55d7a784386ae325dcab3eeb14f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:40.145089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbefd55d7a784386ae325dcab3eeb14f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:40.150748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:40.369697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbefd55d7a784386ae325dcab3eeb14f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:40.448483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbefd55d7a784386ae325dcab3eeb14f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:40.456720 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbefd55d7a784386ae325dcab3eeb14f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:40.457450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbefd55d7a784386ae325dcab3eeb14f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:40.458065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbefd55d7a784386ae325dcab3eeb14f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:40.477922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 897/1795] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:39 2022] DELETE /identity/v3/users/bbefd55d7a784386ae325dcab3eeb14f => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:40.481619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3ddf4-62f9-4110-bddc-65c2ff8c446b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:40.481875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3ddf4-62f9-4110-bddc-65c2ff8c446b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:40.482255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3ddf4-62f9-4110-bddc-65c2ff8c446b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:40.505457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-34c3ddf4-62f9-4110-bddc-65c2ff8c446b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:40.506592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-34c3ddf4-62f9-4110-bddc-65c2ff8c446b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:40.543447 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:40.576221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 899/1796] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:40.588059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:40.590085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:40.691675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60757491-149c-4243-bb3c-d85c8aca1add admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:40.697083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b170a60-fa2f-4271-9770-005a0184bb5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:40.697843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b170a60-fa2f-4271-9770-005a0184bb5a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:40.698189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b170a60-fa2f-4271-9770-005a0184bb5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:40.698543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b170a60-fa2f-4271-9770-005a0184bb5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:40.714020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b170a60-fa2f-4271-9770-005a0184bb5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:40.715840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b170a60-fa2f-4271-9770-005a0184bb5a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:40.729861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 898/1797] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:41.309103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3ddf4-62f9-4110-bddc-65c2ff8c446b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:41.311496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-34c3ddf4-62f9-4110-bddc-65c2ff8c446b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:41.345052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3ddf4-62f9-4110-bddc-65c2ff8c446b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:41.347658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:41.348716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:41.349160 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:41.349746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] PATH_INFO: `/v3/projects/f45909ec6958425f8ff329a028e95ba5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:41.351045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f45909ec6958425f8ff329a028e95ba5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:41.354707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:41.508918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f45909ec6958425f8ff329a028e95ba5', 'request_id': 'req-103ef0da-b11e-44e5-840f-0298aff89520'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:41.515576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f45909ec6958425f8ff329a028e95ba5', 'request_id': 'req-103ef0da-b11e-44e5-840f-0298aff89520'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:41.574788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f45909ec6958425f8ff329a028e95ba5', 'request_id': 'req-103ef0da-b11e-44e5-840f-0298aff89520'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:41.586833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] The token cache is being invalidate because project f45909ec6958425f8ff329a028e95ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:41.587142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:41.588340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:41.597023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 900/1798] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:41 2022] DELETE /identity/v3/projects/f45909ec6958425f8ff329a028e95ba5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:41.597198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b170a60-fa2f-4271-9770-005a0184bb5a service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:41.608243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b170a60-fa2f-4271-9770-005a0184bb5a service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:41.690150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:41.690817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:41.691091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:41.691354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] PATH_INFO: `/v3/users/e0b7ff6d253249a892837e03dade4aba` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:41.701022 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b7ff6d253249a892837e03dade4aba)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:41.703492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:41.724168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:41.953828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b7ff6d253249a892837e03dade4aba'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.065906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b7ff6d253249a892837e03dade4aba'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.074194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b7ff6d253249a892837e03dade4aba'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.075187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b7ff6d253249a892837e03dade4aba'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.075799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-67903b0a-5dbc-461f-8174-4aa6027afc34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b7ff6d253249a892837e03dade4aba'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.095022 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 899/1799] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:41 2022] DELETE /identity/v3/users/e0b7ff6d253249a892837e03dade4aba => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:09:42.569492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:42.571242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:42.585754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ef0da-b11e-44e5-840f-0298aff89520 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:42.644935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:42.645493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:42.645773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:42.646047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] PATH_INFO: `/v3/projects/13136c5b088c4a1994408eb1781d4be4` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:42.647154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13136c5b088c4a1994408eb1781d4be4)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:42.649100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:42.745734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13136c5b088c4a1994408eb1781d4be4', 'request_id': 'req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.751275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13136c5b088c4a1994408eb1781d4be4', 'request_id': 'req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.754318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:42.843347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13136c5b088c4a1994408eb1781d4be4', 'request_id': 'req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.858136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] The token cache is being invalidate because project 13136c5b088c4a1994408eb1781d4be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:09:42.858658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.859642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b22caf67-92e7-4f1a-8b63-ebe5a4fe6030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:42.861903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 901/1800] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:09:42 2022] DELETE /identity/v3/projects/13136c5b088c4a1994408eb1781d4be4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:42.866650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad594365-21cc-4350-99a7-73db2fbe15fb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:42.867111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad594365-21cc-4350-99a7-73db2fbe15fb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:42.867209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad594365-21cc-4350-99a7-73db2fbe15fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:42.895443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ad594365-21cc-4350-99a7-73db2fbe15fb None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:42.896601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ad594365-21cc-4350-99a7-73db2fbe15fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:42.984929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 900/1801] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:42.992872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ae1ed-4127-4ab0-b2f5-32af0710b344 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:42.993156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ae1ed-4127-4ab0-b2f5-32af0710b344 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:42.993432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ae1ed-4127-4ab0-b2f5-32af0710b344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:43.017598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d81ae1ed-4127-4ab0-b2f5-32af0710b344 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:43.019089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d81ae1ed-4127-4ab0-b2f5-32af0710b344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:43.033019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 902/1802] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:43.034142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad594365-21cc-4350-99a7-73db2fbe15fb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:43.044654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad594365-21cc-4350-99a7-73db2fbe15fb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:43.049597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c736-9f36-444e-9fcc-c90bbf46508c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:43.050272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c736-9f36-444e-9fcc-c90bbf46508c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:43.050636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c736-9f36-444e-9fcc-c90bbf46508c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:43.050968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c736-9f36-444e-9fcc-c90bbf46508c admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:43.051708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617c736-9f36-444e-9fcc-c90bbf46508c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:09:43.052116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617c736-9f36-444e-9fcc-c90bbf46508c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:43.053775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617c736-9f36-444e-9fcc-c90bbf46508c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:43.069736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 901/1803] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:09:43.070875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ae1ed-4127-4ab0-b2f5-32af0710b344 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:43.081078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ae1ed-4127-4ab0-b2f5-32af0710b344 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:43.085670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e5b7-c8da-4f25-9aa5-001dd0b73939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:43.086259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e5b7-c8da-4f25-9aa5-001dd0b73939 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:43.086531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e5b7-c8da-4f25-9aa5-001dd0b73939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:43.087106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e5b7-c8da-4f25-9aa5-001dd0b73939 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:43.087741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3e5b7-c8da-4f25-9aa5-001dd0b73939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:43.089804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3e5b7-c8da-4f25-9aa5-001dd0b73939 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:43.197059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 903/1804] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:43 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:43.207726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c736-9f36-444e-9fcc-c90bbf46508c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:43.218411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c736-9f36-444e-9fcc-c90bbf46508c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:43.297611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a22126-ea0c-4c40-a815-4cc4cdc24078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:43.298402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22126-ea0c-4c40-a815-4cc4cdc24078 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:43.298773 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22126-ea0c-4c40-a815-4cc4cdc24078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:43.299143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22126-ea0c-4c40-a815-4cc4cdc24078 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:43.300026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a22126-ea0c-4c40-a815-4cc4cdc24078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:43.302173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a22126-ea0c-4c40-a815-4cc4cdc24078 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:43.387026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 902/1805] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:43 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:43.391331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e5b7-c8da-4f25-9aa5-001dd0b73939 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:43.401944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e5b7-c8da-4f25-9aa5-001dd0b73939 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:43.406535 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e77f6b-1702-486f-9c52-eb22485fce36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:43.407166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e77f6b-1702-486f-9c52-eb22485fce36 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:43.407553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e77f6b-1702-486f-9c52-eb22485fce36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:43.407681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e77f6b-1702-486f-9c52-eb22485fce36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:43.408319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e77f6b-1702-486f-9c52-eb22485fce36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:43.409951 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e77f6b-1702-486f-9c52-eb22485fce36 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:43.421242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 904/1806] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:43.425369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a22126-ea0c-4c40-a815-4cc4cdc24078 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:43.432839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:43.436331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a22126-ea0c-4c40-a815-4cc4cdc24078 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:43.441363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333fb9d-d34c-4b06-9f9a-3b79d0b9d1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:43.441996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333fb9d-d34c-4b06-9f9a-3b79d0b9d1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:43.442262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333fb9d-d34c-4b06-9f9a-3b79d0b9d1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:43.442531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333fb9d-d34c-4b06-9f9a-3b79d0b9d1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:43.443199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333fb9d-d34c-4b06-9f9a-3b79d0b9d1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:43.444808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333fb9d-d34c-4b06-9f9a-3b79d0b9d1c0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:43.459402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 903/1807] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:43.461211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e77f6b-1702-486f-9c52-eb22485fce36 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:43.473414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e77f6b-1702-486f-9c52-eb22485fce36 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:43.478067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbe80e-3902-480b-b126-d2a7d7b30032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:43.479370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbe80e-3902-480b-b126-d2a7d7b30032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:43.479669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbe80e-3902-480b-b126-d2a7d7b30032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:43.479917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbe80e-3902-480b-b126-d2a7d7b30032 admin admin] PATH_INFO: `/v3/projects/51e564f2dbcb44eb80c54a76579d7896/users/01e0007c06e64b4b979b888cd7d42c96/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:43.498019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbe80e-3902-480b-b126-d2a7d7b30032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e564f2dbcb44eb80c54a76579d7896, user_id=01e0007c06e64b4b979b888cd7d42c96, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:43.500882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbe80e-3902-480b-b126-d2a7d7b30032 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:43.609111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 905/1808] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:43 2022] PUT /identity/v3/projects/51e564f2dbcb44eb80c54a76579d7896/users/01e0007c06e64b4b979b888cd7d42c96/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:43.612006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e944b9-7f59-4626-a23a-0804c9692f3a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:43.612297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e944b9-7f59-4626-a23a-0804c9692f3a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:43.612635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e944b9-7f59-4626-a23a-0804c9692f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:43.650831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f6e944b9-7f59-4626-a23a-0804c9692f3a None None] MFA Rules not processed for user `01e0007c06e64b4b979b888cd7d42c96`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:43.652336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f6e944b9-7f59-4626-a23a-0804c9692f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:43.776181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 904/1809] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:43.780441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbe80e-3902-480b-b126-d2a7d7b30032 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:43.860857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbe80e-3902-480b-b126-d2a7d7b30032 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:43.865687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33169850-4ba0-46d3-afa2-0b34384392c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:43.866285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33169850-4ba0-46d3-afa2-0b34384392c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:43.866557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33169850-4ba0-46d3-afa2-0b34384392c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:43.866820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33169850-4ba0-46d3-afa2-0b34384392c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:43.867437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33169850-4ba0-46d3-afa2-0b34384392c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:43.870029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33169850-4ba0-46d3-afa2-0b34384392c5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:43.975683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 906/1810] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:43 2022] POST /identity/v3/projects => generated 387 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:43.980349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e944b9-7f59-4626-a23a-0804c9692f3a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:43.993337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e944b9-7f59-4626-a23a-0804c9692f3a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:44.057718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142598fe-4843-4ad1-bc4a-d31e321a05af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:44.058697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142598fe-4843-4ad1-bc4a-d31e321a05af admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:44.061288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142598fe-4843-4ad1-bc4a-d31e321a05af admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:44.061288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142598fe-4843-4ad1-bc4a-d31e321a05af admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:44.061288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142598fe-4843-4ad1-bc4a-d31e321a05af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:44.061967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142598fe-4843-4ad1-bc4a-d31e321a05af admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:44.140208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 905/1811] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:43 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:44.144135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33169850-4ba0-46d3-afa2-0b34384392c5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:44.154360 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33169850-4ba0-46d3-afa2-0b34384392c5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:44.158884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f7c54-21a7-4b09-9550-1a68150112b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:44.159513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f7c54-21a7-4b09-9550-1a68150112b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:44.159780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f7c54-21a7-4b09-9550-1a68150112b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:44.165121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f7c54-21a7-4b09-9550-1a68150112b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:44.165121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f7c54-21a7-4b09-9550-1a68150112b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:44.165121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f7c54-21a7-4b09-9550-1a68150112b0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:44.174265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 907/1812] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:44.178597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142598fe-4843-4ad1-bc4a-d31e321a05af admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:44.189217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142598fe-4843-4ad1-bc4a-d31e321a05af admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:44.193921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65d22c-ef36-4cee-9f4f-42f596f0196a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:44.194599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65d22c-ef36-4cee-9f4f-42f596f0196a admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:44.194879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65d22c-ef36-4cee-9f4f-42f596f0196a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:44.195149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65d22c-ef36-4cee-9f4f-42f596f0196a admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:44.195787 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65d22c-ef36-4cee-9f4f-42f596f0196a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:44.197946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65d22c-ef36-4cee-9f4f-42f596f0196a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:44.210045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 906/1813] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:09:44.214208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f7c54-21a7-4b09-9550-1a68150112b0 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:44.225832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f7c54-21a7-4b09-9550-1a68150112b0 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:44.230199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489186f0-d984-43cc-ae2c-3c73579a7e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:44.230845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489186f0-d984-43cc-ae2c-3c73579a7e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:44.231175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489186f0-d984-43cc-ae2c-3c73579a7e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:44.231547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489186f0-d984-43cc-ae2c-3c73579a7e56 admin admin] PATH_INFO: `/v3/projects/0f9c623533c849d48e7dc8c346fedfdc/users/ac1c59ac48a240cda9a7db4c8a152bca/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:44.262390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489186f0-d984-43cc-ae2c-3c73579a7e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9c623533c849d48e7dc8c346fedfdc, user_id=ac1c59ac48a240cda9a7db4c8a152bca, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:44.266277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489186f0-d984-43cc-ae2c-3c73579a7e56 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:44.326550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 908/1814] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:44 2022] PUT /identity/v3/projects/0f9c623533c849d48e7dc8c346fedfdc/users/ac1c59ac48a240cda9a7db4c8a152bca/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:09:44.337099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f671e3e-f0df-481a-864d-e1559bebf7e0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:44.337406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f671e3e-f0df-481a-864d-e1559bebf7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:44.337684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f671e3e-f0df-481a-864d-e1559bebf7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:44.371880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6f671e3e-f0df-481a-864d-e1559bebf7e0 None None] MFA Rules not processed for user `ac1c59ac48a240cda9a7db4c8a152bca`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:44.373413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6f671e3e-f0df-481a-864d-e1559bebf7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:44.444698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:44.473330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 907/1815] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:09:44.476418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489186f0-d984-43cc-ae2c-3c73579a7e56 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:44.523923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:44.550128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489186f0-d984-43cc-ae2c-3c73579a7e56 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:44.554984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbaffd-9e12-45f8-8b24-389176609d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:44.555612 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbaffd-9e12-45f8-8b24-389176609d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:44.556087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbaffd-9e12-45f8-8b24-389176609d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:44.556751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbaffd-9e12-45f8-8b24-389176609d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:44.557343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedbaffd-9e12-45f8-8b24-389176609d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:44.559241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedbaffd-9e12-45f8-8b24-389176609d60 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:44.662231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f671e3e-f0df-481a-864d-e1559bebf7e0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:44.664420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 909/1816] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:09:44 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:09:44.677721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f671e3e-f0df-481a-864d-e1559bebf7e0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:44.771721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8f80a-22fa-40f7-932a-2fc982e1b948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:44.772585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8f80a-22fa-40f7-932a-2fc982e1b948 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:44.773095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8f80a-22fa-40f7-932a-2fc982e1b948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:44.773470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8f80a-22fa-40f7-932a-2fc982e1b948 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:44.774199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8f80a-22fa-40f7-932a-2fc982e1b948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:44.776999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8f80a-22fa-40f7-932a-2fc982e1b948 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:44.940207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 908/1817] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:09:44 2022] POST /identity/v3/users => generated 300 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:09:44.940547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbaffd-9e12-45f8-8b24-389176609d60 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:44.952109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbaffd-9e12-45f8-8b24-389176609d60 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:44.958027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d59d1c-4404-4839-8958-b3d724045404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:44.958825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59d1c-4404-4839-8958-b3d724045404 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:44.959220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59d1c-4404-4839-8958-b3d724045404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:44.959666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59d1c-4404-4839-8958-b3d724045404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:44.960459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d59d1c-4404-4839-8958-b3d724045404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:44.964805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d59d1c-4404-4839-8958-b3d724045404 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:44.975548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 910/1818] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:09:44.981718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8f80a-22fa-40f7-932a-2fc982e1b948 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:44.993558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8f80a-22fa-40f7-932a-2fc982e1b948 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:44.999114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a61c5-e3d4-4be4-b4f2-e8ba07cd061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:45.000220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a61c5-e3d4-4be4-b4f2-e8ba07cd061b admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:45.000588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a61c5-e3d4-4be4-b4f2-e8ba07cd061b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:45.000929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a61c5-e3d4-4be4-b4f2-e8ba07cd061b admin admin] PATH_INFO: `/v3/projects/25af57a6f58649acb597485d16250239/users/367855eeeae141f795a060a82b082b08/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:45.018140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a61c5-e3d4-4be4-b4f2-e8ba07cd061b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25af57a6f58649acb597485d16250239, user_id=367855eeeae141f795a060a82b082b08, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:45.021127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a61c5-e3d4-4be4-b4f2-e8ba07cd061b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:45.149468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 909/1819] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:44 2022] PUT /identity/v3/projects/25af57a6f58649acb597485d16250239/users/367855eeeae141f795a060a82b082b08/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:45.153481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d59d1c-4404-4839-8958-b3d724045404 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:45.165847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d59d1c-4404-4839-8958-b3d724045404 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:45.243192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495a425-0802-49be-aecf-c38bface63d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:45.244240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495a425-0802-49be-aecf-c38bface63d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:45.244567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495a425-0802-49be-aecf-c38bface63d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:45.244815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495a425-0802-49be-aecf-c38bface63d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:45.245450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495a425-0802-49be-aecf-c38bface63d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:45.247069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495a425-0802-49be-aecf-c38bface63d0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:45.257003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 911/1820] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:09:45.261707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a61c5-e3d4-4be4-b4f2-e8ba07cd061b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:45.273123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a61c5-e3d4-4be4-b4f2-e8ba07cd061b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:45.278284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05aad1-93a5-485a-a337-5654db931b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:45.278928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05aad1-93a5-485a-a337-5654db931b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:45.279200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05aad1-93a5-485a-a337-5654db931b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:45.279467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05aad1-93a5-485a-a337-5654db931b7f admin admin] PATH_INFO: `/v3/projects/25af57a6f58649acb597485d16250239/users/367855eeeae141f795a060a82b082b08/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:45.281393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05aad1-93a5-485a-a337-5654db931b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25af57a6f58649acb597485d16250239, user_id=367855eeeae141f795a060a82b082b08, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:45.284026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05aad1-93a5-485a-a337-5654db931b7f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:45.317789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 910/1821] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:09:45 2022] PUT /identity/v3/projects/25af57a6f58649acb597485d16250239/users/367855eeeae141f795a060a82b082b08/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:45.329711 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec6cf-8b6b-4db5-82a7-5c1d9514a317 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:45.330266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec6cf-8b6b-4db5-82a7-5c1d9514a317 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:45.330671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec6cf-8b6b-4db5-82a7-5c1d9514a317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:45.360813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b16ec6cf-8b6b-4db5-82a7-5c1d9514a317 None None] MFA Rules not processed for user `367855eeeae141f795a060a82b082b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:45.362187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b16ec6cf-8b6b-4db5-82a7-5c1d9514a317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:45.463855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 912/1822] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:09:45.504736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05aad1-93a5-485a-a337-5654db931b7f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:45.506432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-de05aad1-93a5-485a-a337-5654db931b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:45.587479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05aad1-93a5-485a-a337-5654db931b7f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:45.592113 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff735d1-ffbf-4265-93b0-66cd2f9c57ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:45.592790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff735d1-ffbf-4265-93b0-66cd2f9c57ea service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:45.593075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff735d1-ffbf-4265-93b0-66cd2f9c57ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:45.593333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff735d1-ffbf-4265-93b0-66cd2f9c57ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:45.604184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff735d1-ffbf-4265-93b0-66cd2f9c57ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:45.605840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff735d1-ffbf-4265-93b0-66cd2f9c57ea service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:45.630943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 911/1823] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:46.928303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ec6cf-8b6b-4db5-82a7-5c1d9514a317 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:46.944667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ec6cf-8b6b-4db5-82a7-5c1d9514a317 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:46.948808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6346bd1-f492-497c-a921-5ed9081ad1fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:46.949664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6346bd1-f492-497c-a921-5ed9081ad1fc service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:46.949938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6346bd1-f492-497c-a921-5ed9081ad1fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:46.950219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6346bd1-f492-497c-a921-5ed9081ad1fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:46.983963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:47.021019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6346bd1-f492-497c-a921-5ed9081ad1fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:47.021849 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6346bd1-f492-497c-a921-5ed9081ad1fc service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:47.032227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 913/1824] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:55.613240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:56.305972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff735d1-ffbf-4265-93b0-66cd2f9c57ea service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:56.317539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff735d1-ffbf-4265-93b0-66cd2f9c57ea service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:56.326341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f401836-d602-49f1-b5ac-d6f2f5bebd47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:56.327083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f401836-d602-49f1-b5ac-d6f2f5bebd47 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:56.327359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f401836-d602-49f1-b5ac-d6f2f5bebd47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:56.327620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f401836-d602-49f1-b5ac-d6f2f5bebd47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:56.408714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f401836-d602-49f1-b5ac-d6f2f5bebd47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:56.410732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f401836-d602-49f1-b5ac-d6f2f5bebd47 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:56.420631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 912/1825] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:09:57.076771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=14 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:09:58.831198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6346bd1-f492-497c-a921-5ed9081ad1fc service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:58.832712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f6346bd1-f492-497c-a921-5ed9081ad1fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:58.921358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6346bd1-f492-497c-a921-5ed9081ad1fc service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:58.934336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:58.935440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:58.950550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:58.955061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] PATH_INFO: `/v3/users/f880bd118d3b4048954f975d3d907bf8` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:58.959374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f880bd118d3b4048954f975d3d907bf8)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:58.962844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:59.287283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f880bd118d3b4048954f975d3d907bf8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:59.367656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f880bd118d3b4048954f975d3d907bf8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:59.377545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f880bd118d3b4048954f975d3d907bf8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:59.378307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f880bd118d3b4048954f975d3d907bf8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:59.378922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f880bd118d3b4048954f975d3d907bf8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:09:59.414377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 914/1826] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:09:58 2022] DELETE /identity/v3/users/f880bd118d3b4048954f975d3d907bf8 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:09:59.419499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0da2e-945d-4bc7-8ab9-14069d4d7016 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:59.419989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0da2e-945d-4bc7-8ab9-14069d4d7016 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:59.420084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0da2e-945d-4bc7-8ab9-14069d4d7016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:59.441869 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-afa0da2e-945d-4bc7-8ab9-14069d4d7016 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:09:59.443336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-afa0da2e-945d-4bc7-8ab9-14069d4d7016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:59.511991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 913/1827] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:09:59.522735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:09:59.524306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:09:59.598082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e8db9-0d0a-450d-b741-5759f9a83597 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:09:59.603271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec64c6-b247-457d-9fff-488b688c7289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:09:59.604168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec64c6-b247-457d-9fff-488b688c7289 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:09:59.610960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec64c6-b247-457d-9fff-488b688c7289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:09:59.610960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec64c6-b247-457d-9fff-488b688c7289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:09:59.615224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec64c6-b247-457d-9fff-488b688c7289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:09:59.616862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec64c6-b247-457d-9fff-488b688c7289 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:09:59.628860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 915/1828] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:00.337262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0da2e-945d-4bc7-8ab9-14069d4d7016 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:00.339646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-afa0da2e-945d-4bc7-8ab9-14069d4d7016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:00.360185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0da2e-945d-4bc7-8ab9-14069d4d7016 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:00.366809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:00.367465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:00.367746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:00.374386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] PATH_INFO: `/v3/projects/16af2915d7254078ad58c7592a20f126` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:00.375847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16af2915d7254078ad58c7592a20f126)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:00.382723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:00.548971 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16af2915d7254078ad58c7592a20f126', 'request_id': 'req-2218dafd-2342-4889-b996-005496252dc3'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:00.579804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16af2915d7254078ad58c7592a20f126', 'request_id': 'req-2218dafd-2342-4889-b996-005496252dc3'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:00.671019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16af2915d7254078ad58c7592a20f126', 'request_id': 'req-2218dafd-2342-4889-b996-005496252dc3'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:00.703901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] The token cache is being invalidate because project 16af2915d7254078ad58c7592a20f126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:00.704085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:00.705148 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:00.708338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 914/1829] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:00 2022] DELETE /identity/v3/projects/16af2915d7254078ad58c7592a20f126 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:00.711767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec64c6-b247-457d-9fff-488b688c7289 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:00.723351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec64c6-b247-457d-9fff-488b688c7289 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:00.745385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:00.787448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:00.788334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:00.788653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:00.788924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] PATH_INFO: `/v3/users/6255db3e42844d6388f6018ef6b3c03f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:00.790149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6255db3e42844d6388f6018ef6b3c03f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:00.792087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:00.975881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6255db3e42844d6388f6018ef6b3c03f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.099673 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6255db3e42844d6388f6018ef6b3c03f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.111999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6255db3e42844d6388f6018ef6b3c03f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.113577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6255db3e42844d6388f6018ef6b3c03f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.114233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6255db3e42844d6388f6018ef6b3c03f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.146584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 916/1830] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:00 2022] DELETE /identity/v3/users/6255db3e42844d6388f6018ef6b3c03f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:01.447094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:01.448563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:01.474649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218dafd-2342-4889-b996-005496252dc3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:01.516596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:01.595288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:01.595902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:01.595902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:01.596075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] PATH_INFO: `/v3/users/01e0007c06e64b4b979b888cd7d42c96` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:01.597651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e0007c06e64b4b979b888cd7d42c96)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:01.599916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:01.730829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:01.732364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:01.751570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4388bd-12fe-480b-b948-5ac15cd75bc3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:01.759888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:01.761202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:01.761628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:01.762046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] PATH_INFO: `/v3/projects/fa30ab0e6e924bed808ce918e654e8e2` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:01.763586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa30ab0e6e924bed808ce918e654e8e2)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:01.765984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:01.829998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e0007c06e64b4b979b888cd7d42c96'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.918795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa30ab0e6e924bed808ce918e654e8e2', 'request_id': 'req-11c28793-9f70-4f8f-8d6d-11e1d29e638f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.924753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa30ab0e6e924bed808ce918e654e8e2', 'request_id': 'req-11c28793-9f70-4f8f-8d6d-11e1d29e638f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.933779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e0007c06e64b4b979b888cd7d42c96'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.942797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e0007c06e64b4b979b888cd7d42c96'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.944240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e0007c06e64b4b979b888cd7d42c96'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.944896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e0007c06e64b4b979b888cd7d42c96'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:01.964886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 915/1831] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:01 2022] DELETE /identity/v3/users/01e0007c06e64b4b979b888cd7d42c96 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:01.966247 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:01.967138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.013958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa30ab0e6e924bed808ce918e654e8e2', 'request_id': 'req-11c28793-9f70-4f8f-8d6d-11e1d29e638f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.027274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] The token cache is being invalidate because project fa30ab0e6e924bed808ce918e654e8e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:02.027576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.028304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-11c28793-9f70-4f8f-8d6d-11e1d29e638f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.030199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 917/1832] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:01 2022] DELETE /identity/v3/projects/fa30ab0e6e924bed808ce918e654e8e2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:02.034620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b0fcd-ffb5-453a-ae27-e012b78d1a20 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.034889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b0fcd-ffb5-453a-ae27-e012b78d1a20 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.035157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b0fcd-ffb5-453a-ae27-e012b78d1a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.039419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3023ea4-96fe-4df5-b444-e97022d26f0b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.044280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.044957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.051100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.051100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] PATH_INFO: `/v3/users/ec887671b602492f817f229baf8a8803` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.051100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec887671b602492f817f229baf8a8803)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.051100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.056937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b47b0fcd-ffb5-453a-ae27-e012b78d1a20 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:02.058153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b47b0fcd-ffb5-453a-ae27-e012b78d1a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.069372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 918/1833] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:02.070610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfbf15-0ddc-4893-a8c5-51e5b6b20567 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.070888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfbf15-0ddc-4893-a8c5-51e5b6b20567 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.071154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfbf15-0ddc-4893-a8c5-51e5b6b20567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.093677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8ccfbf15-0ddc-4893-a8c5-51e5b6b20567 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:02.095090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8ccfbf15-0ddc-4893-a8c5-51e5b6b20567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.103373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 919/1834] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:02.104115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfbf15-0ddc-4893-a8c5-51e5b6b20567 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.105604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8ccfbf15-0ddc-4893-a8c5-51e5b6b20567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.118302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:02.161418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfbf15-0ddc-4893-a8c5-51e5b6b20567 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.165237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e0a16-2b03-4ec5-b154-d141f916d944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.165832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e0a16-2b03-4ec5-b154-d141f916d944 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.166135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e0a16-2b03-4ec5-b154-d141f916d944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.166433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e0a16-2b03-4ec5-b154-d141f916d944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.175236 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e0a16-2b03-4ec5-b154-d141f916d944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.176688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e0a16-2b03-4ec5-b154-d141f916d944 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.186249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 920/1835] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:02.187381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d40db0-4398-4cf6-bb32-16983ba2ffd3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.187661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d40db0-4398-4cf6-bb32-16983ba2ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.187962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d40db0-4398-4cf6-bb32-16983ba2ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.207878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-28d40db0-4398-4cf6-bb32-16983ba2ffd3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:02.209128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-28d40db0-4398-4cf6-bb32-16983ba2ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.216999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 921/1836] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:02.221104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d40db0-4398-4cf6-bb32-16983ba2ffd3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.229862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d40db0-4398-4cf6-bb32-16983ba2ffd3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.234668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69e848-5f1f-4737-a642-d248b05f1512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.235248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69e848-5f1f-4737-a642-d248b05f1512 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.235539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69e848-5f1f-4737-a642-d248b05f1512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.235786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69e848-5f1f-4737-a642-d248b05f1512 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.236421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69e848-5f1f-4737-a642-d248b05f1512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:02.236745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69e848-5f1f-4737-a642-d248b05f1512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.238630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69e848-5f1f-4737-a642-d248b05f1512 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.250922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 922/1837] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:10:02.256298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69e848-5f1f-4737-a642-d248b05f1512 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.268764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69e848-5f1f-4737-a642-d248b05f1512 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.273608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e843bc6-3b73-4a21-b310-029c269de318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.274689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843bc6-3b73-4a21-b310-029c269de318 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.274689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843bc6-3b73-4a21-b310-029c269de318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.274947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843bc6-3b73-4a21-b310-029c269de318 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.275343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e843bc6-3b73-4a21-b310-029c269de318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.277308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e843bc6-3b73-4a21-b310-029c269de318 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.305292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:02.308395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec887671b602492f817f229baf8a8803'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.385654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec887671b602492f817f229baf8a8803'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.396954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 923/1838] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:02 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:02.398191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e843bc6-3b73-4a21-b310-029c269de318 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.398311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec887671b602492f817f229baf8a8803'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.399434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec887671b602492f817f229baf8a8803'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.400051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b9210821-6f82-4b42-a227-e73f0b3466bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec887671b602492f817f229baf8a8803'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.421337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 916/1839] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:01 2022] DELETE /identity/v3/users/ec887671b602492f817f229baf8a8803 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:02.425891 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c01b91-e9af-4494-b415-ea5d5e690150 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.426206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c01b91-e9af-4494-b415-ea5d5e690150 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.426473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c01b91-e9af-4494-b415-ea5d5e690150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.448270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d5c01b91-e9af-4494-b415-ea5d5e690150 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:02.450324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d5c01b91-e9af-4494-b415-ea5d5e690150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.474259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e843bc6-3b73-4a21-b310-029c269de318 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.520420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 917/1840] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:02.531164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c01b91-e9af-4494-b415-ea5d5e690150 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.532720 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d5c01b91-e9af-4494-b415-ea5d5e690150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.539753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485a75e-ef6d-48c7-9830-c15a6203176e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.540630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485a75e-ef6d-48c7-9830-c15a6203176e admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.540970 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485a75e-ef6d-48c7-9830-c15a6203176e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.541771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485a75e-ef6d-48c7-9830-c15a6203176e admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.542543 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485a75e-ef6d-48c7-9830-c15a6203176e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.544882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485a75e-ef6d-48c7-9830-c15a6203176e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.597196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c01b91-e9af-4494-b415-ea5d5e690150 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.600936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc430a7-f6d2-4860-8bc4-b7bc6986cf55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.601538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc430a7-f6d2-4860-8bc4-b7bc6986cf55 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.601811 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc430a7-f6d2-4860-8bc4-b7bc6986cf55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.602079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc430a7-f6d2-4860-8bc4-b7bc6986cf55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.611595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc430a7-f6d2-4860-8bc4-b7bc6986cf55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.613048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc430a7-f6d2-4860-8bc4-b7bc6986cf55 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.622659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 918/1841] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:02.640133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 924/1842] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:02 2022] POST /identity/v3/users => generated 291 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:02.641551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc430a7-f6d2-4860-8bc4-b7bc6986cf55 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.642938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-efc430a7-f6d2-4860-8bc4-b7bc6986cf55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.657185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc430a7-f6d2-4860-8bc4-b7bc6986cf55 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.662250 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b7f44-3cca-4bea-b781-80d0c4795dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.662864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b7f44-3cca-4bea-b781-80d0c4795dde admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.663140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b7f44-3cca-4bea-b781-80d0c4795dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.663398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b7f44-3cca-4bea-b781-80d0c4795dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.664250 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b7f44-3cca-4bea-b781-80d0c4795dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.665698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b7f44-3cca-4bea-b781-80d0c4795dde admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.666151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485a75e-ef6d-48c7-9830-c15a6203176e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.671883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9485a75e-ef6d-48c7-9830-c15a6203176e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.679675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 919/1843] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:02.681640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b7f44-3cca-4bea-b781-80d0c4795dde admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.695292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b7f44-3cca-4bea-b781-80d0c4795dde admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.703362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fad718-7789-4486-a1af-67fccdb6a8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.703599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fad718-7789-4486-a1af-67fccdb6a8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.703877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fad718-7789-4486-a1af-67fccdb6a8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.704147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fad718-7789-4486-a1af-67fccdb6a8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.704988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485a75e-ef6d-48c7-9830-c15a6203176e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.705834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fad718-7789-4486-a1af-67fccdb6a8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.707443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fad718-7789-4486-a1af-67fccdb6a8c8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.710070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.710781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.711115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.711447 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] PATH_INFO: `/v3/projects/51e564f2dbcb44eb80c54a76579d7896` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.712736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e564f2dbcb44eb80c54a76579d7896)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.715318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.718485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:02.724564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 920/1844] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:02.732746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fad718-7789-4486-a1af-67fccdb6a8c8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.742916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fad718-7789-4486-a1af-67fccdb6a8c8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:02.747562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbd015-0c1e-457c-ab0d-b794c17dc4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:02.748194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbd015-0c1e-457c-ab0d-b794c17dc4a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.748476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbd015-0c1e-457c-ab0d-b794c17dc4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.748779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbd015-0c1e-457c-ab0d-b794c17dc4a1 admin admin] PATH_INFO: `/v3/projects/2f1a54eb45644c91b07c17bfb1ca13ac/users/4adae5d710734e6bbaf7bf87a83084e5/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.764527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbd015-0c1e-457c-ab0d-b794c17dc4a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1a54eb45644c91b07c17bfb1ca13ac, user_id=4adae5d710734e6bbaf7bf87a83084e5, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:02.767274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbd015-0c1e-457c-ab0d-b794c17dc4a1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:02.884229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 921/1845] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:02 2022] PUT /identity/v3/projects/2f1a54eb45644c91b07c17bfb1ca13ac/users/4adae5d710734e6bbaf7bf87a83084e5/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:02.896007 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f718b2-c52e-474b-b61c-fec450be378a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:02.896756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f718b2-c52e-474b-b61c-fec450be378a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:02.897124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f718b2-c52e-474b-b61c-fec450be378a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:02.903046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e564f2dbcb44eb80c54a76579d7896', 'request_id': 'req-a5e25c11-870e-4e39-a490-c1da2751a523'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.910972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e564f2dbcb44eb80c54a76579d7896', 'request_id': 'req-a5e25c11-870e-4e39-a490-c1da2751a523'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.927547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-27f718b2-c52e-474b-b61c-fec450be378a None None] MFA Rules not processed for user `4adae5d710734e6bbaf7bf87a83084e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:02.928876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-27f718b2-c52e-474b-b61c-fec450be378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:02.934795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:02.978811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e564f2dbcb44eb80c54a76579d7896', 'request_id': 'req-a5e25c11-870e-4e39-a490-c1da2751a523'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.991118 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] The token cache is being invalidate because project 51e564f2dbcb44eb80c54a76579d7896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:02.991393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.992061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:02.993303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 922/1846] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:02.993729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 925/1847] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:02 2022] DELETE /identity/v3/projects/51e564f2dbcb44eb80c54a76579d7896 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:02.998081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.998226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f718b2-c52e-474b-b61c-fec450be378a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:02.999532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:03.067901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f718b2-c52e-474b-b61c-fec450be378a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:03.073537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25c11-870e-4e39-a490-c1da2751a523 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:03.073828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:03.074753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:03.075030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:03.075294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] PATH_INFO: `/v3/users/ac1c59ac48a240cda9a7db4c8a152bca` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:03.076948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1c59ac48a240cda9a7db4c8a152bca)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:03.078911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267c44f-2c8c-4e8b-8368-411c0c1f573b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:03.079517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267c44f-2c8c-4e8b-8368-411c0c1f573b admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:03.079768 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:03.079880 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267c44f-2c8c-4e8b-8368-411c0c1f573b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:03.080049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267c44f-2c8c-4e8b-8368-411c0c1f573b admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:03.080740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267c44f-2c8c-4e8b-8368-411c0c1f573b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:03.082828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267c44f-2c8c-4e8b-8368-411c0c1f573b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:03.170223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 926/1848] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:02 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:03.171465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267c44f-2c8c-4e8b-8368-411c0c1f573b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:03.176130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c267c44f-2c8c-4e8b-8368-411c0c1f573b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:03.245951 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267c44f-2c8c-4e8b-8368-411c0c1f573b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:03.251367 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:03.251997 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:03.252280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:03.254552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] PATH_INFO: `/v3/projects/a395d0a9a4f44df09daaaeb1201cd9c9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:03.254552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a395d0a9a4f44df09daaaeb1201cd9c9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:03.255647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:03.265847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1c59ac48a240cda9a7db4c8a152bca'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.280432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:03.328084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1c59ac48a240cda9a7db4c8a152bca'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.336868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1c59ac48a240cda9a7db4c8a152bca'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.337795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1c59ac48a240cda9a7db4c8a152bca'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.338418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1c59ac48a240cda9a7db4c8a152bca'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.356808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 923/1849] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:02 2022] DELETE /identity/v3/users/ac1c59ac48a240cda9a7db4c8a152bca => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:03.357766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:03.359560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:03.375988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a395d0a9a4f44df09daaaeb1201cd9c9', 'request_id': 'req-2a341696-f628-4683-acd9-a1d29bd4478e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.386553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a395d0a9a4f44df09daaaeb1201cd9c9', 'request_id': 'req-2a341696-f628-4683-acd9-a1d29bd4478e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.394846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:03.430917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a395d0a9a4f44df09daaaeb1201cd9c9', 'request_id': 'req-2a341696-f628-4683-acd9-a1d29bd4478e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.433321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b47fe2-87b1-439f-acfe-68b23b0cfc0e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:03.438960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaf632-0ba1-4457-b8e9-fde033c1ac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:03.439576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaf632-0ba1-4457-b8e9-fde033c1ac67 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:03.439855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaf632-0ba1-4457-b8e9-fde033c1ac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:03.440263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaf632-0ba1-4457-b8e9-fde033c1ac67 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:03.442465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfaf632-0ba1-4457-b8e9-fde033c1ac67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:03.444025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] The token cache is being invalidate because project a395d0a9a4f44df09daaaeb1201cd9c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:03.444326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.444708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfaf632-0ba1-4457-b8e9-fde033c1ac67 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:03.445026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.446822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 927/1850] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:03 2022] DELETE /identity/v3/projects/a395d0a9a4f44df09daaaeb1201cd9c9 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:03.451506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:03.452959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:03.523390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a341696-f628-4683-acd9-a1d29bd4478e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:03.528749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:03.529374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:03.529647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:03.529915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] PATH_INFO: `/v3/users/a3b928284cf64d2da7f0ae58f59f3939` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:03.531108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b928284cf64d2da7f0ae58f59f3939)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:03.533094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:03.580832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 924/1851] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:03 2022] POST /identity/v3/users => generated 296 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:03.584817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaf632-0ba1-4457-b8e9-fde033c1ac67 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:03.586619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ddfaf632-0ba1-4457-b8e9-fde033c1ac67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:03.605362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaf632-0ba1-4457-b8e9-fde033c1ac67 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:03.605924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867624d-b44f-4ad2-8aec-edd895fe3f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:03.606573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867624d-b44f-4ad2-8aec-edd895fe3f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:03.606857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867624d-b44f-4ad2-8aec-edd895fe3f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:03.607133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867624d-b44f-4ad2-8aec-edd895fe3f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:03.607772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867624d-b44f-4ad2-8aec-edd895fe3f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:03.609442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867624d-b44f-4ad2-8aec-edd895fe3f35 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:03.626580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 925/1852] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:03.627972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867624d-b44f-4ad2-8aec-edd895fe3f35 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:03.640824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867624d-b44f-4ad2-8aec-edd895fe3f35 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:03.645592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406a886-2a32-4ffb-87f6-7261bb33949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:03.646237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406a886-2a32-4ffb-87f6-7261bb33949d admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:03.646511 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406a886-2a32-4ffb-87f6-7261bb33949d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:03.646781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406a886-2a32-4ffb-87f6-7261bb33949d admin admin] PATH_INFO: `/v3/projects/96630614449049feb0a6c15606c500d2/users/535ecd30f79c4571853e9f6534188c6c/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:03.662756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406a886-2a32-4ffb-87f6-7261bb33949d admin admin] RBAC: Authorizing `identity:create_grant(project_id=96630614449049feb0a6c15606c500d2, user_id=535ecd30f79c4571853e9f6534188c6c, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:03.667096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406a886-2a32-4ffb-87f6-7261bb33949d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:03.751446 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 926/1853] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:03 2022] PUT /identity/v3/projects/96630614449049feb0a6c15606c500d2/users/535ecd30f79c4571853e9f6534188c6c/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:03.755345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406a886-2a32-4ffb-87f6-7261bb33949d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:03.796809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b928284cf64d2da7f0ae58f59f3939'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.850299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406a886-2a32-4ffb-87f6-7261bb33949d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:03.855494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc8234-ddf2-41e8-af06-c493f71e640f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:03.855954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc8234-ddf2-41e8-af06-c493f71e640f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:03.856234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc8234-ddf2-41e8-af06-c493f71e640f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:03.856495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc8234-ddf2-41e8-af06-c493f71e640f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:03.857168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc8234-ddf2-41e8-af06-c493f71e640f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:03.858867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc8234-ddf2-41e8-af06-c493f71e640f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:03.865046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b928284cf64d2da7f0ae58f59f3939'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.874999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 927/1854] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:03.875590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b928284cf64d2da7f0ae58f59f3939'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.876357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b928284cf64d2da7f0ae58f59f3939'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.877057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc8234-ddf2-41e8-af06-c493f71e640f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:03.877333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b928284cf64d2da7f0ae58f59f3939'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:03.880356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-55dc8234-ddf2-41e8-af06-c493f71e640f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:03.898731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc8234-ddf2-41e8-af06-c493f71e640f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:03.906852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:03.907961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:03.908771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:03.909249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] PATH_INFO: `/v3/projects/0f9c623533c849d48e7dc8c346fedfdc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:03.911032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f9c623533c849d48e7dc8c346fedfdc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:03.916754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 928/1855] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:03 2022] DELETE /identity/v3/users/a3b928284cf64d2da7f0ae58f59f3939 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:03.916754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:03.918449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:03.925748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:03.990166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:04.033899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1e0e2-3ab6-4784-8d4e-7fc4ef224c66 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:04.041943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858b1e-f329-4282-a51f-730df4747179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:04.043204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858b1e-f329-4282-a51f-730df4747179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:04.043664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858b1e-f329-4282-a51f-730df4747179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:04.044136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858b1e-f329-4282-a51f-730df4747179 admin admin] PATH_INFO: `/v3/projects/96630614449049feb0a6c15606c500d2/users/535ecd30f79c4571853e9f6534188c6c/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:04.054909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858b1e-f329-4282-a51f-730df4747179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96630614449049feb0a6c15606c500d2, user_id=535ecd30f79c4571853e9f6534188c6c, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:04.054909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858b1e-f329-4282-a51f-730df4747179 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:04.064096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 929/1856] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:03 2022] PUT /identity/v3/projects/96630614449049feb0a6c15606c500d2/users/535ecd30f79c4571853e9f6534188c6c/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:04.076628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cf2d8-5089-4484-82f4-bcfc8eccd4ea None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:04.077123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cf2d8-5089-4484-82f4-bcfc8eccd4ea None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:04.077833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cf2d8-5089-4484-82f4-bcfc8eccd4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:04.094559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f9c623533c849d48e7dc8c346fedfdc', 'request_id': 'req-471874f9-fca2-477b-a093-86257c3b54a8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.101545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f9c623533c849d48e7dc8c346fedfdc', 'request_id': 'req-471874f9-fca2-477b-a093-86257c3b54a8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.109587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8d1cf2d8-5089-4484-82f4-bcfc8eccd4ea None None] MFA Rules not processed for user `535ecd30f79c4571853e9f6534188c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:04.110800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8d1cf2d8-5089-4484-82f4-bcfc8eccd4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:04.196557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 930/1857] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:04.210384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cf2d8-5089-4484-82f4-bcfc8eccd4ea None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:04.211786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8d1cf2d8-5089-4484-82f4-bcfc8eccd4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:04.230655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f9c623533c849d48e7dc8c346fedfdc', 'request_id': 'req-471874f9-fca2-477b-a093-86257c3b54a8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.245765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:04.250182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] The token cache is being invalidate because project 0f9c623533c849d48e7dc8c346fedfdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:04.250472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.251578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.253312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 928/1858] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:03 2022] DELETE /identity/v3/projects/0f9c623533c849d48e7dc8c346fedfdc => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:04.257069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:04.258455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:04.283564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cf2d8-5089-4484-82f4-bcfc8eccd4ea None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:04.287349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:04.287958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:04.288233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:04.288498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:04.290373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:04.305262 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:04.306734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:04.316896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 931/1859] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:04.332250 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471874f9-fca2-477b-a093-86257c3b54a8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:04.411181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:04.411965 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:04.412248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:04.412614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] PATH_INFO: `/v3/users/367855eeeae141f795a060a82b082b08` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:04.413816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=367855eeeae141f795a060a82b082b08)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:04.415743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:04.445577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:04.447866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:04.465674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc826811-7ebb-4c44-b3ba-5b34fc913904 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:04.471829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:04.472469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:04.472815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:04.473090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] PATH_INFO: `/v3/projects/c72ef3caaa6449ee81ca7a08f7f55d6d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:04.474237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c72ef3caaa6449ee81ca7a08f7f55d6d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:04.476380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:04.696879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367855eeeae141f795a060a82b082b08'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.702891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c72ef3caaa6449ee81ca7a08f7f55d6d', 'request_id': 'req-673df5ff-817b-457f-8016-f047ddaa3254'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.708874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c72ef3caaa6449ee81ca7a08f7f55d6d', 'request_id': 'req-673df5ff-817b-457f-8016-f047ddaa3254'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.762175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367855eeeae141f795a060a82b082b08'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.763279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:04.770066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367855eeeae141f795a060a82b082b08'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.770840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367855eeeae141f795a060a82b082b08'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.771485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5db1d7f7-94c8-4bde-bf83-1ec8a1488e89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367855eeeae141f795a060a82b082b08'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.789166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 929/1860] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:04 2022] DELETE /identity/v3/users/367855eeeae141f795a060a82b082b08 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:04.819521 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c72ef3caaa6449ee81ca7a08f7f55d6d', 'request_id': 'req-673df5ff-817b-457f-8016-f047ddaa3254'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.832635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] The token cache is being invalidate because project c72ef3caaa6449ee81ca7a08f7f55d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:04.832939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.833964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-673df5ff-817b-457f-8016-f047ddaa3254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:04.838689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 932/1861] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:04 2022] DELETE /identity/v3/projects/c72ef3caaa6449ee81ca7a08f7f55d6d => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:04.840713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8ad68-e78a-482b-8862-40210f2426ee None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:04.840953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8ad68-e78a-482b-8862-40210f2426ee None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:04.841300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8ad68-e78a-482b-8862-40210f2426ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:04.862414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9fd8ad68-e78a-482b-8862-40210f2426ee None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:04.863606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9fd8ad68-e78a-482b-8862-40210f2426ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:04.926816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 930/1862] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:04.936366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41de95-ffac-4d40-b3fe-bf850fb9e217 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:04.936891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41de95-ffac-4d40-b3fe-bf850fb9e217 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:04.936981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41de95-ffac-4d40-b3fe-bf850fb9e217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:04.958255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3d41de95-ffac-4d40-b3fe-bf850fb9e217 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:04.959529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3d41de95-ffac-4d40-b3fe-bf850fb9e217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:04.967642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 933/1863] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:04.971944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8ad68-e78a-482b-8862-40210f2426ee None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:04.982489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8ad68-e78a-482b-8862-40210f2426ee None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:04.987368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ec908-165f-478d-8f2b-c9047c77c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:04.988479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ec908-165f-478d-8f2b-c9047c77c4bb admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:04.988479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ec908-165f-478d-8f2b-c9047c77c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:04.988773 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ec908-165f-478d-8f2b-c9047c77c4bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:04.989960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ec908-165f-478d-8f2b-c9047c77c4bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:04.990259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ec908-165f-478d-8f2b-c9047c77c4bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:04.991871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ec908-165f-478d-8f2b-c9047c77c4bb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.010516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41de95-ffac-4d40-b3fe-bf850fb9e217 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.012955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 931/1864] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:05.022168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41de95-ffac-4d40-b3fe-bf850fb9e217 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.026608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483998a6-e211-460e-88f1-2397794b5536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.027222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483998a6-e211-460e-88f1-2397794b5536 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.027561 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483998a6-e211-460e-88f1-2397794b5536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.027835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483998a6-e211-460e-88f1-2397794b5536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.028457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483998a6-e211-460e-88f1-2397794b5536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.030616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483998a6-e211-460e-88f1-2397794b5536 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.171625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 934/1865] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:05 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:05.173018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ec908-165f-478d-8f2b-c9047c77c4bb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.183895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ec908-165f-478d-8f2b-c9047c77c4bb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.246792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03666163-47bd-4e1d-a54e-d93d0c3aeacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.247711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03666163-47bd-4e1d-a54e-d93d0c3aeacb admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.247711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03666163-47bd-4e1d-a54e-d93d0c3aeacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.247920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03666163-47bd-4e1d-a54e-d93d0c3aeacb admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.248860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03666163-47bd-4e1d-a54e-d93d0c3aeacb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.250505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03666163-47bd-4e1d-a54e-d93d0c3aeacb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.272057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483998a6-e211-460e-88f1-2397794b5536 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.273905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-483998a6-e211-460e-88f1-2397794b5536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:05.293777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483998a6-e211-460e-88f1-2397794b5536 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.303013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.303013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.303013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.303013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] PATH_INFO: `/v3/projects/25af57a6f58649acb597485d16250239` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.303469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25af57a6f58649acb597485d16250239)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.304587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.352557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 932/1866] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:05 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:05.357437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03666163-47bd-4e1d-a54e-d93d0c3aeacb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.368204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03666163-47bd-4e1d-a54e-d93d0c3aeacb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.373784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a827e51-3127-454a-be98-41766b653f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.374071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a827e51-3127-454a-be98-41766b653f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.374348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a827e51-3127-454a-be98-41766b653f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.374615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a827e51-3127-454a-be98-41766b653f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.375369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a827e51-3127-454a-be98-41766b653f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.377003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a827e51-3127-454a-be98-41766b653f7e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.391790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 933/1867] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:05.392371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a827e51-3127-454a-be98-41766b653f7e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.402774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a827e51-3127-454a-be98-41766b653f7e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.416859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25af57a6f58649acb597485d16250239', 'request_id': 'req-af274110-fe19-4c10-b432-0fbbcabcd3f5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:05.428008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25af57a6f58649acb597485d16250239', 'request_id': 'req-af274110-fe19-4c10-b432-0fbbcabcd3f5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:05.464904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644e9f-4959-4529-84cc-c70b841efbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.465541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644e9f-4959-4529-84cc-c70b841efbcd admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.465817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644e9f-4959-4529-84cc-c70b841efbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.466181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644e9f-4959-4529-84cc-c70b841efbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.466823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a644e9f-4959-4529-84cc-c70b841efbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.468465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a644e9f-4959-4529-84cc-c70b841efbcd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.482293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 934/1868] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:05.482293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644e9f-4959-4529-84cc-c70b841efbcd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.482293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:05.486292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25af57a6f58649acb597485d16250239', 'request_id': 'req-af274110-fe19-4c10-b432-0fbbcabcd3f5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:05.486382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4a644e9f-4959-4529-84cc-c70b841efbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:05.495265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] The token cache is being invalidate because project 25af57a6f58649acb597485d16250239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:05.495539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:05.496212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:05.502322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 935/1869] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:05 2022] DELETE /identity/v3/projects/25af57a6f58649acb597485d16250239 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:05.503212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.504843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:05.521962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af274110-fe19-4c10-b432-0fbbcabcd3f5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.527846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459afd6-b28a-4bda-9f02-8638030072af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.528846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459afd6-b28a-4bda-9f02-8638030072af admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.529048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459afd6-b28a-4bda-9f02-8638030072af admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.529442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459afd6-b28a-4bda-9f02-8638030072af admin admin] PATH_INFO: `/v3/projects/2a32d20bc716430f935b3a72166d04cd/users/ae4efcb9cbcd4661b087f0ba99b52f01/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.548401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459afd6-b28a-4bda-9f02-8638030072af admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a32d20bc716430f935b3a72166d04cd, user_id=ae4efcb9cbcd4661b087f0ba99b52f01, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.552398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459afd6-b28a-4bda-9f02-8638030072af admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.557555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644e9f-4959-4529-84cc-c70b841efbcd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.562165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122669e-103a-4b1f-b4ec-9bbaab71c443 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535ecd30f79c4571853e9f6534188c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96630614449049feb0a6c15606c500d2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.562765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122669e-103a-4b1f-b4ec-9bbaab71c443 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.563042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122669e-103a-4b1f-b4ec-9bbaab71c443 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.563308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122669e-103a-4b1f-b4ec-9bbaab71c443 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.563942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122669e-103a-4b1f-b4ec-9bbaab71c443 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:05.564238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122669e-103a-4b1f-b4ec-9bbaab71c443 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.565837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122669e-103a-4b1f-b4ec-9bbaab71c443 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.578739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 935/1870] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:05.579922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a095cfa-2845-4a30-bcbd-ea9840a06fb3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.580199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a095cfa-2845-4a30-bcbd-ea9840a06fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.580464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a095cfa-2845-4a30-bcbd-ea9840a06fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.599417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2a095cfa-2845-4a30-bcbd-ea9840a06fb3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:05.600608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2a095cfa-2845-4a30-bcbd-ea9840a06fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:05.608333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 936/1871] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:05.609085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a095cfa-2845-4a30-bcbd-ea9840a06fb3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.610287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2a095cfa-2845-4a30-bcbd-ea9840a06fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:05.630930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a095cfa-2845-4a30-bcbd-ea9840a06fb3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.640059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29522a20-f258-4326-96b4-94a5f25b7dfd tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535ecd30f79c4571853e9f6534188c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96630614449049feb0a6c15606c500d2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.640960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29522a20-f258-4326-96b4-94a5f25b7dfd tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.641307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29522a20-f258-4326-96b4-94a5f25b7dfd tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.641656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29522a20-f258-4326-96b4-94a5f25b7dfd tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.642362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29522a20-f258-4326-96b4-94a5f25b7dfd tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.644345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29522a20-f258-4326-96b4-94a5f25b7dfd tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.674309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 936/1872] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:05 2022] PUT /identity/v3/projects/2a32d20bc716430f935b3a72166d04cd/users/ae4efcb9cbcd4661b087f0ba99b52f01/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:05.676206 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5118dc3-33cf-43e2-b674-686d558ad4a9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.676699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5118dc3-33cf-43e2-b674-686d558ad4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.677152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5118dc3-33cf-43e2-b674-686d558ad4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.703176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a5118dc3-33cf-43e2-b674-686d558ad4a9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:05.704883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a5118dc3-33cf-43e2-b674-686d558ad4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:05.783456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 937/1873] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:05.790314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca44c9-3ca5-4d8d-bf72-a8169b4461fe None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.790886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca44c9-3ca5-4d8d-bf72-a8169b4461fe None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.791776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca44c9-3ca5-4d8d-bf72-a8169b4461fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.799326 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 937/1874] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:05 2022] POST /identity/v3/projects => generated 354 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:05.800806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29522a20-f258-4326-96b4-94a5f25b7dfd tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.839214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-31ca44c9-3ca5-4d8d-bf72-a8169b4461fe None None] MFA Rules not processed for user `ae4efcb9cbcd4661b087f0ba99b52f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:05.841686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-31ca44c9-3ca5-4d8d-bf72-a8169b4461fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:05.902653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29522a20-f258-4326-96b4-94a5f25b7dfd tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.908855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424266e-5a56-434f-adae-4ccc8b13d6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.909785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424266e-5a56-434f-adae-4ccc8b13d6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.910333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424266e-5a56-434f-adae-4ccc8b13d6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.912062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424266e-5a56-434f-adae-4ccc8b13d6c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.913305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424266e-5a56-434f-adae-4ccc8b13d6c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:05.913771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424266e-5a56-434f-adae-4ccc8b13d6c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.915672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424266e-5a56-434f-adae-4ccc8b13d6c0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:05.934203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 938/1875] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:05.935063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424266e-5a56-434f-adae-4ccc8b13d6c0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.940788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 938/1876] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:05.941654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca44c9-3ca5-4d8d-bf72-a8169b4461fe None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:05.955677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca44c9-3ca5-4d8d-bf72-a8169b4461fe None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:05.962017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc98e0-7878-42d7-8c5a-9addd31fa906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:05.964392 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:05.964902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc98e0-7878-42d7-8c5a-9addd31fa906 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:05.965350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc98e0-7878-42d7-8c5a-9addd31fa906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:05.969489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc98e0-7878-42d7-8c5a-9addd31fa906 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:05.969489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc98e0-7878-42d7-8c5a-9addd31fa906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:05.969489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc98e0-7878-42d7-8c5a-9addd31fa906 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.011933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424266e-5a56-434f-adae-4ccc8b13d6c0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.018000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc10367-40c8-4137-bb35-dc7e609704f7 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535ecd30f79c4571853e9f6534188c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96630614449049feb0a6c15606c500d2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.018664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc10367-40c8-4137-bb35-dc7e609704f7 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.018981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc10367-40c8-4137-bb35-dc7e609704f7 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.019222 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc10367-40c8-4137-bb35-dc7e609704f7 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.019857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc10367-40c8-4137-bb35-dc7e609704f7 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:06.020152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc10367-40c8-4137-bb35-dc7e609704f7 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.021764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc10367-40c8-4137-bb35-dc7e609704f7 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.033703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 939/1877] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:06.034567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc10367-40c8-4137-bb35-dc7e609704f7 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.048357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc10367-40c8-4137-bb35-dc7e609704f7 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.054520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0bfce-545a-4ead-a19b-3f58d109bda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.055310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0bfce-545a-4ead-a19b-3f58d109bda7 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.055588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0bfce-545a-4ead-a19b-3f58d109bda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.055857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0bfce-545a-4ead-a19b-3f58d109bda7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.057267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0bfce-545a-4ead-a19b-3f58d109bda7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.059318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0bfce-545a-4ead-a19b-3f58d109bda7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.076208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 939/1878] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:05 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:06.097872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc98e0-7878-42d7-8c5a-9addd31fa906 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.165419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc98e0-7878-42d7-8c5a-9addd31fa906 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.170036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874424f-c80a-455a-93ef-b644ba6e98a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.170636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874424f-c80a-455a-93ef-b644ba6e98a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.170911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874424f-c80a-455a-93ef-b644ba6e98a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.171179 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874424f-c80a-455a-93ef-b644ba6e98a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.171788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874424f-c80a-455a-93ef-b644ba6e98a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.173760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874424f-c80a-455a-93ef-b644ba6e98a7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.209311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 940/1879] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:06 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:06.210196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0bfce-545a-4ead-a19b-3f58d109bda7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.274476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0bfce-545a-4ead-a19b-3f58d109bda7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.279381 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535ecd30f79c4571853e9f6534188c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96630614449049feb0a6c15606c500d2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.280185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.280330 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.280633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] PATH_INFO: `/v3/projects/35f50077266f49f88158d4a1b3900d1e` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.281789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Authorizing `identity:delete_project(project_id=35f50077266f49f88158d4a1b3900d1e)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.283703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.288293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 940/1880] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:06 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:06.288293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874424f-c80a-455a-93ef-b644ba6e98a7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.299466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874424f-c80a-455a-93ef-b644ba6e98a7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.356197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afd2cc-38cb-43fb-aaa8-72bc2e5ac3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.356827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afd2cc-38cb-43fb-aaa8-72bc2e5ac3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.357104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afd2cc-38cb-43fb-aaa8-72bc2e5ac3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.357372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afd2cc-38cb-43fb-aaa8-72bc2e5ac3b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.358012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afd2cc-38cb-43fb-aaa8-72bc2e5ac3b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.359920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afd2cc-38cb-43fb-aaa8-72bc2e5ac3b0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.396756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f50077266f49f88158d4a1b3900d1e', 'request_id': 'req-c220f2c9-756c-4e5a-913b-acff17a48730'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:06.402182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f50077266f49f88158d4a1b3900d1e', 'request_id': 'req-c220f2c9-756c-4e5a-913b-acff17a48730'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:06.487539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 941/1881] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:06 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:06.488357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afd2cc-38cb-43fb-aaa8-72bc2e5ac3b0 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.498169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afd2cc-38cb-43fb-aaa8-72bc2e5ac3b0 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.516817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f50077266f49f88158d4a1b3900d1e', 'request_id': 'req-c220f2c9-756c-4e5a-913b-acff17a48730'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:06.528981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] The token cache is being invalidate because project 35f50077266f49f88158d4a1b3900d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:06.529266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:06.529919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:06.531656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 941/1882] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:06 2022] DELETE /identity/v3/projects/35f50077266f49f88158d4a1b3900d1e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:06.532550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.533959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:06.556351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5584874-c4fe-42b8-a9e0-d9f555585daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.557695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5584874-c4fe-42b8-a9e0-d9f555585daf admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.558096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5584874-c4fe-42b8-a9e0-d9f555585daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.558472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5584874-c4fe-42b8-a9e0-d9f555585daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.559893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5584874-c4fe-42b8-a9e0-d9f555585daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.561677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5584874-c4fe-42b8-a9e0-d9f555585daf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.578385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 942/1883] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:06.579301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5584874-c4fe-42b8-a9e0-d9f555585daf admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.581120 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b5584874-c4fe-42b8-a9e0-d9f555585daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:06.595727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5584874-c4fe-42b8-a9e0-d9f555585daf admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.603973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:06.604121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd17dd-1232-4d68-ab56-5c409b1561db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.604121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd17dd-1232-4d68-ab56-5c409b1561db admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.604121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd17dd-1232-4d68-ab56-5c409b1561db admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.604121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd17dd-1232-4d68-ab56-5c409b1561db admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.604453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd17dd-1232-4d68-ab56-5c409b1561db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.604569 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220f2c9-756c-4e5a-913b-acff17a48730 tempest-VolumeQuotaClassesTest-998474499 tempest-VolumeQuotaClassesTest-998474499-project-admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.606099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd17dd-1232-4d68-ab56-5c409b1561db admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.608929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8afb2-b966-4174-b928-d3dfa6e7bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.609845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8afb2-b966-4174-b928-d3dfa6e7bb72 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.610312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8afb2-b966-4174-b928-d3dfa6e7bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.610584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8afb2-b966-4174-b928-d3dfa6e7bb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.611235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8afb2-b966-4174-b928-d3dfa6e7bb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.612843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8afb2-b966-4174-b928-d3dfa6e7bb72 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.620546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 943/1884] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:06.622742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd17dd-1232-4d68-ab56-5c409b1561db admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.623902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 942/1885] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:06.631601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8afb2-b966-4174-b928-d3dfa6e7bb72 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.632670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd17dd-1232-4d68-ab56-5c409b1561db admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.637431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30b452-a021-4466-82cd-eed8c5f41617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.638092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30b452-a021-4466-82cd-eed8c5f41617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.638360 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30b452-a021-4466-82cd-eed8c5f41617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.638625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30b452-a021-4466-82cd-eed8c5f41617 admin admin] PATH_INFO: `/v3/projects/08c7180c708c4b2cb4c42cb40596503e/users/e2c78172690e4e619e6193ffb7bdd5aa/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.642047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8afb2-b966-4174-b928-d3dfa6e7bb72 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.646715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a4730-240c-4068-a098-ad86052d9094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.647352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a4730-240c-4068-a098-ad86052d9094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.647627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a4730-240c-4068-a098-ad86052d9094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.647893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a4730-240c-4068-a098-ad86052d9094 admin admin] PATH_INFO: `/v3/projects/9686e187429a43cfbec8d9ce03d8fe9b/users/3a29696fc3374d8c9e454aa8219d36d4/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.653615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30b452-a021-4466-82cd-eed8c5f41617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c7180c708c4b2cb4c42cb40596503e, user_id=e2c78172690e4e619e6193ffb7bdd5aa, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.657200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30b452-a021-4466-82cd-eed8c5f41617 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.663174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a4730-240c-4068-a098-ad86052d9094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9686e187429a43cfbec8d9ce03d8fe9b, user_id=3a29696fc3374d8c9e454aa8219d36d4, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.666496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a4730-240c-4068-a098-ad86052d9094 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.758151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 944/1886] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:06 2022] PUT /identity/v3/projects/08c7180c708c4b2cb4c42cb40596503e/users/e2c78172690e4e619e6193ffb7bdd5aa/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:06.758713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 943/1887] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:06 2022] PUT /identity/v3/projects/9686e187429a43cfbec8d9ce03d8fe9b/users/3a29696fc3374d8c9e454aa8219d36d4/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:06.762569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30b452-a021-4466-82cd-eed8c5f41617 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.773641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1df5f-77d7-4b19-8db3-c5ededc81e3c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.773923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1df5f-77d7-4b19-8db3-c5ededc81e3c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.774387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1df5f-77d7-4b19-8db3-c5ededc81e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.815006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a7a1df5f-77d7-4b19-8db3-c5ededc81e3c None None] MFA Rules not processed for user `e2c78172690e4e619e6193ffb7bdd5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:06.816217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a7a1df5f-77d7-4b19-8db3-c5ededc81e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:06.837091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30b452-a021-4466-82cd-eed8c5f41617 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.859765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:06.883801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:06.894716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 944/1888] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:06.901067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d2556-1a7a-4052-9e61-339ebf477bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.901687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d2556-1a7a-4052-9e61-339ebf477bba admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.901952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d2556-1a7a-4052-9e61-339ebf477bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.902218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d2556-1a7a-4052-9e61-339ebf477bba admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.902850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d2556-1a7a-4052-9e61-339ebf477bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.904480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d2556-1a7a-4052-9e61-339ebf477bba admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.906324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1df5f-77d7-4b19-8db3-c5ededc81e3c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.908301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a7a1df5f-77d7-4b19-8db3-c5ededc81e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:06.914466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 945/1889] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:06.921939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d2556-1a7a-4052-9e61-339ebf477bba admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:06.933328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d2556-1a7a-4052-9e61-339ebf477bba admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:06.938251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4597373-806f-4da1-a6cc-bef4f5bdc280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:06.938898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4597373-806f-4da1-a6cc-bef4f5bdc280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.939173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4597373-806f-4da1-a6cc-bef4f5bdc280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.939443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4597373-806f-4da1-a6cc-bef4f5bdc280 admin admin] PATH_INFO: `/v3/projects/9686e187429a43cfbec8d9ce03d8fe9b/users/3a29696fc3374d8c9e454aa8219d36d4/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.941478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4597373-806f-4da1-a6cc-bef4f5bdc280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9686e187429a43cfbec8d9ce03d8fe9b, user_id=3a29696fc3374d8c9e454aa8219d36d4, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:06.944118 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4597373-806f-4da1-a6cc-bef4f5bdc280 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:06.959943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 946/1890] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:06 2022] PUT /identity/v3/projects/9686e187429a43cfbec8d9ce03d8fe9b/users/3a29696fc3374d8c9e454aa8219d36d4/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:06.975208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078cb7-7b43-4338-94db-fc15e6422ec0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:06.975208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078cb7-7b43-4338-94db-fc15e6422ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:06.975208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078cb7-7b43-4338-94db-fc15e6422ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:06.981041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1df5f-77d7-4b19-8db3-c5ededc81e3c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:07.005707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-39078cb7-7b43-4338-94db-fc15e6422ec0 None None] MFA Rules not processed for user `3a29696fc3374d8c9e454aa8219d36d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:07.006831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-39078cb7-7b43-4338-94db-fc15e6422ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:07.028129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c455d9-302c-4b2e-b5f0-f946ec1c239c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:07.028801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c455d9-302c-4b2e-b5f0-f946ec1c239c service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:07.029078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c455d9-302c-4b2e-b5f0-f946ec1c239c service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:07.029939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c455d9-302c-4b2e-b5f0-f946ec1c239c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:07.079803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 947/1891] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:07.088875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c455d9-302c-4b2e-b5f0-f946ec1c239c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:07.090346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c455d9-302c-4b2e-b5f0-f946ec1c239c service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:07.100274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 945/1892] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:07.111723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39078cb7-7b43-4338-94db-fc15e6422ec0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:07.112474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-39078cb7-7b43-4338-94db-fc15e6422ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:07.128077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39078cb7-7b43-4338-94db-fc15e6422ec0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:07.131635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c218a9-7950-4afa-be9f-ca1588527648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:07.132358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c218a9-7950-4afa-be9f-ca1588527648 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:07.132668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c218a9-7950-4afa-be9f-ca1588527648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:07.132927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c218a9-7950-4afa-be9f-ca1588527648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:07.143273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c218a9-7950-4afa-be9f-ca1588527648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:07.144796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c218a9-7950-4afa-be9f-ca1588527648 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:07.156225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 948/1893] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:07.188078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c455d9-302c-4b2e-b5f0-f946ec1c239c service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:07.218398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c218a9-7950-4afa-be9f-ca1588527648 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:07.220307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f4c218a9-7950-4afa-be9f-ca1588527648 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:07.237015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:07.276974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:07.283797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c455d9-302c-4b2e-b5f0-f946ec1c239c service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:07.290535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:07.291955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:07.292233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:07.292523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] PATH_INFO: `/v3/users/ae4efcb9cbcd4661b087f0ba99b52f01` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:07.293713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae4efcb9cbcd4661b087f0ba99b52f01)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:07.295653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:07.303509 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c218a9-7950-4afa-be9f-ca1588527648 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:07.309338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:07.310270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:07.313153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:07.313153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] PATH_INFO: `/v3/users/4adae5d710734e6bbaf7bf87a83084e5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:07.313153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4adae5d710734e6bbaf7bf87a83084e5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:07.314370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:07.484186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae4efcb9cbcd4661b087f0ba99b52f01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.515395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4adae5d710734e6bbaf7bf87a83084e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.556020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae4efcb9cbcd4661b087f0ba99b52f01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.563624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae4efcb9cbcd4661b087f0ba99b52f01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.564276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae4efcb9cbcd4661b087f0ba99b52f01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.565765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8564b79a-1946-4b93-acdf-366dbbe9c0c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae4efcb9cbcd4661b087f0ba99b52f01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.582671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 946/1894] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:07 2022] DELETE /identity/v3/users/ae4efcb9cbcd4661b087f0ba99b52f01 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:07.587146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28715f00-1c35-49b7-abcc-765dc83ea34c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:07.588152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28715f00-1c35-49b7-abcc-765dc83ea34c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:07.588454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28715f00-1c35-49b7-abcc-765dc83ea34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:07.609146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-28715f00-1c35-49b7-abcc-765dc83ea34c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:07.610380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-28715f00-1c35-49b7-abcc-765dc83ea34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:07.613163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4adae5d710734e6bbaf7bf87a83084e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.614732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:07.621981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4adae5d710734e6bbaf7bf87a83084e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.622764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4adae5d710734e6bbaf7bf87a83084e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.623582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b7194d6d-cb61-4984-bd2d-b5345ee38080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4adae5d710734e6bbaf7bf87a83084e5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:07.641553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 949/1895] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:07 2022] DELETE /identity/v3/users/4adae5d710734e6bbaf7bf87a83084e5 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:07.645815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a9baa-7f48-494d-aae7-8493313e4615 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:07.646169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a9baa-7f48-494d-aae7-8493313e4615 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:07.646518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a9baa-7f48-494d-aae7-8493313e4615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:07.666587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ee3a9baa-7f48-494d-aae7-8493313e4615 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:07.668432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ee3a9baa-7f48-494d-aae7-8493313e4615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:07.681590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 947/1896] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:07.691432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28715f00-1c35-49b7-abcc-765dc83ea34c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:07.693575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-28715f00-1c35-49b7-abcc-765dc83ea34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:07.737974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 950/1897] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:07.747852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a9baa-7f48-494d-aae7-8493313e4615 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:07.749305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ee3a9baa-7f48-494d-aae7-8493313e4615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:07.755820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28715f00-1c35-49b7-abcc-765dc83ea34c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:07.782906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:07.806358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3dfdc-c84e-4e7c-ba29-1f97a93f5946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:07.807012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3dfdc-c84e-4e7c-ba29-1f97a93f5946 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:07.807274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3dfdc-c84e-4e7c-ba29-1f97a93f5946 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:07.807580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3dfdc-c84e-4e7c-ba29-1f97a93f5946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:07.809464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a9baa-7f48-494d-aae7-8493313e4615 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:07.813599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e32a9-9305-4827-9450-9597b4419fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:07.818315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e32a9-9305-4827-9450-9597b4419fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:07.818315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e32a9-9305-4827-9450-9597b4419fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:07.818315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e32a9-9305-4827-9450-9597b4419fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:07.818583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3dfdc-c84e-4e7c-ba29-1f97a93f5946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:07.819338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3dfdc-c84e-4e7c-ba29-1f97a93f5946 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:07.825701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e32a9-9305-4827-9450-9597b4419fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:07.827138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e32a9-9305-4827-9450-9597b4419fe5 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:07.844775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 948/1898] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:07.844899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 951/1899] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:07.967853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:08.491210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e32a9-9305-4827-9450-9597b4419fe5 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:08.493328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-da3e32a9-9305-4827-9450-9597b4419fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:08.493452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3dfdc-c84e-4e7c-ba29-1f97a93f5946 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:08.494772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d7a3dfdc-c84e-4e7c-ba29-1f97a93f5946 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:08.530080 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e32a9-9305-4827-9450-9597b4419fe5 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:08.531397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3dfdc-c84e-4e7c-ba29-1f97a93f5946 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:08.543146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:08.547066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:08.547066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:08.547066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] PATH_INFO: `/v3/projects/2a32d20bc716430f935b3a72166d04cd` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:08.549942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:08.550565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:08.550914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:08.551082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] PATH_INFO: `/v3/projects/2f1a54eb45644c91b07c17bfb1ca13ac` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:08.552290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1a54eb45644c91b07c17bfb1ca13ac)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:08.553858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a32d20bc716430f935b3a72166d04cd)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:08.554368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:08.558627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:08.722339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a32d20bc716430f935b3a72166d04cd', 'request_id': 'req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.722719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1a54eb45644c91b07c17bfb1ca13ac', 'request_id': 'req-7c5848a1-ce98-415c-a3d5-c7297b1f1303'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.728023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a32d20bc716430f935b3a72166d04cd', 'request_id': 'req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.728589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1a54eb45644c91b07c17bfb1ca13ac', 'request_id': 'req-7c5848a1-ce98-415c-a3d5-c7297b1f1303'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.876266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a32d20bc716430f935b3a72166d04cd', 'request_id': 'req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.884202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:08.888963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] The token cache is being invalidate because project 2a32d20bc716430f935b3a72166d04cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:08.889254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.892761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.892761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 952/1900] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:08 2022] DELETE /identity/v3/projects/2a32d20bc716430f935b3a72166d04cd => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:08.895459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:08.896795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:08.939056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1a54eb45644c91b07c17bfb1ca13ac', 'request_id': 'req-7c5848a1-ce98-415c-a3d5-c7297b1f1303'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.961957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abc5cb-bfca-4ddc-bb70-af0faa73f86e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:08.966256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] The token cache is being invalidate because project 2f1a54eb45644c91b07c17bfb1ca13ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:08.966546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.967197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:08.973696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 949/1901] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:08 2022] DELETE /identity/v3/projects/2f1a54eb45644c91b07c17bfb1ca13ac => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:08.974730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:08.976095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:09.023868 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:09.024492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:09.024862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:09.025129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] PATH_INFO: `/v3/users/3a29696fc3374d8c9e454aa8219d36d4` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:09.026799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a29696fc3374d8c9e454aa8219d36d4)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:09.028805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:09.043417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5848a1-ce98-415c-a3d5-c7297b1f1303 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:09.047939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:09.048600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:09.048875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:09.049137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] PATH_INFO: `/v3/users/535ecd30f79c4571853e9f6534188c6c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:09.050388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=535ecd30f79c4571853e9f6534188c6c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:09.052286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:09.056897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:09.249841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a29696fc3374d8c9e454aa8219d36d4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.250213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535ecd30f79c4571853e9f6534188c6c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.345004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a29696fc3374d8c9e454aa8219d36d4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.352942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a29696fc3374d8c9e454aa8219d36d4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.353634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a29696fc3374d8c9e454aa8219d36d4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.354271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a29696fc3374d8c9e454aa8219d36d4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.371911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 953/1902] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:08 2022] DELETE /identity/v3/users/3a29696fc3374d8c9e454aa8219d36d4 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:09.436386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535ecd30f79c4571853e9f6534188c6c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.445118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535ecd30f79c4571853e9f6534188c6c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.445872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535ecd30f79c4571853e9f6534188c6c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.446746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535ecd30f79c4571853e9f6534188c6c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:09.469545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 950/1903] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:08 2022] DELETE /identity/v3/users/535ecd30f79c4571853e9f6534188c6c => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:09.926773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:09.928474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:09.968756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:10.005634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5148a-1acc-49d4-a746-118cd5125413 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.060428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.064333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:10.071485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.072199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.072578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.073034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] PATH_INFO: `/v3/projects/9686e187429a43cfbec8d9ce03d8fe9b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.074516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9686e187429a43cfbec8d9ce03d8fe9b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.077522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.083635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345aae-1b4c-49e7-b67c-6b47b6639ddd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.089617 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.090257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.091098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.091098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] PATH_INFO: `/v3/projects/96630614449049feb0a6c15606c500d2` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.092071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96630614449049feb0a6c15606c500d2)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.094380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.238182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9686e187429a43cfbec8d9ce03d8fe9b', 'request_id': 'req-62f3dad9-67f0-4673-978d-9c319daf2cd5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.241427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96630614449049feb0a6c15606c500d2', 'request_id': 'req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.245635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9686e187429a43cfbec8d9ce03d8fe9b', 'request_id': 'req-62f3dad9-67f0-4673-978d-9c319daf2cd5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.247974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96630614449049feb0a6c15606c500d2', 'request_id': 'req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.299318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9686e187429a43cfbec8d9ce03d8fe9b', 'request_id': 'req-62f3dad9-67f0-4673-978d-9c319daf2cd5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.314737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] The token cache is being invalidate because project 9686e187429a43cfbec8d9ce03d8fe9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:10.315025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.315678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-62f3dad9-67f0-4673-978d-9c319daf2cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.317421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 954/1904] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:09 2022] DELETE /identity/v3/projects/9686e187429a43cfbec8d9ce03d8fe9b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:10.325135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15117c05-6222-4660-869e-01a167bf6f26 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.325489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15117c05-6222-4660-869e-01a167bf6f26 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.325833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15117c05-6222-4660-869e-01a167bf6f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.345671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-15117c05-6222-4660-869e-01a167bf6f26 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:10.346947 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-15117c05-6222-4660-869e-01a167bf6f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:10.362074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96630614449049feb0a6c15606c500d2', 'request_id': 'req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.376180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] The token cache is being invalidate because project 96630614449049feb0a6c15606c500d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:10.378100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.378100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6f1cf0bf-a1d6-4bd7-86f1-16b6b792aaf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:10.379015 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 951/1905] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:10 2022] DELETE /identity/v3/projects/96630614449049feb0a6c15606c500d2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:10.384088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ef467-3da7-4582-a576-bc928730a5f9 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.384442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ef467-3da7-4582-a576-bc928730a5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.384810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ef467-3da7-4582-a576-bc928730a5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.406061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-384ef467-3da7-4582-a576-bc928730a5f9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:10.407379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-384ef467-3da7-4582-a576-bc928730a5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:10.424641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 952/1907] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:10.425017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 955/1907] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:10.426624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd721089-51a6-4046-8d79-4f18bfa6cee0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.426912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd721089-51a6-4046-8d79-4f18bfa6cee0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.427173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3503-1b23-4968-bac5-2ceae853b8f5 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.427276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd721089-51a6-4046-8d79-4f18bfa6cee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.427461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3503-1b23-4968-bac5-2ceae853b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.427842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3503-1b23-4968-bac5-2ceae853b8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.448576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-cd721089-51a6-4046-8d79-4f18bfa6cee0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:10.448781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7edc3503-1b23-4968-bac5-2ceae853b8f5 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:10.450117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7edc3503-1b23-4968-bac5-2ceae853b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:10.450281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cd721089-51a6-4046-8d79-4f18bfa6cee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:10.458972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 956/1908] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:10.464157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 953/1909] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:10.464300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd721089-51a6-4046-8d79-4f18bfa6cee0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.465495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc3503-1b23-4968-bac5-2ceae853b8f5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.474251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd721089-51a6-4046-8d79-4f18bfa6cee0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.475309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc3503-1b23-4968-bac5-2ceae853b8f5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.479960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f03fa-a85f-4bbf-bd98-981e9cb5bea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.480618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3e3c-4d1d-4913-824e-484eda7d5952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.480966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f03fa-a85f-4bbf-bd98-981e9cb5bea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.481209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3e3c-4d1d-4913-824e-484eda7d5952 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.481485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3e3c-4d1d-4913-824e-484eda7d5952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.481696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f03fa-a85f-4bbf-bd98-981e9cb5bea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.481806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3e3c-4d1d-4913-824e-484eda7d5952 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.482210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f03fa-a85f-4bbf-bd98-981e9cb5bea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.482390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3e3c-4d1d-4913-824e-484eda7d5952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:10.482691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3e3c-4d1d-4913-824e-484eda7d5952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.483307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f03fa-a85f-4bbf-bd98-981e9cb5bea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:10.483755 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f03fa-a85f-4bbf-bd98-981e9cb5bea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.484322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3e3c-4d1d-4913-824e-484eda7d5952 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.485756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f03fa-a85f-4bbf-bd98-981e9cb5bea5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.496854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 954/1910] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:10:10.498185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 957/1911] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:10:10.502239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3e3c-4d1d-4913-824e-484eda7d5952 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.502522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f03fa-a85f-4bbf-bd98-981e9cb5bea5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.512861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f03fa-a85f-4bbf-bd98-981e9cb5bea5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.513009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3e3c-4d1d-4913-824e-484eda7d5952 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.517574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482d76d-b4f6-463b-89a8-10fbb7dd4ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.518200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482d76d-b4f6-463b-89a8-10fbb7dd4ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.518477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482d76d-b4f6-463b-89a8-10fbb7dd4ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.518747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482d76d-b4f6-463b-89a8-10fbb7dd4ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.519379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482d76d-b4f6-463b-89a8-10fbb7dd4ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.519592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ca70f-eb44-4b86-a941-cd393688d71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.520382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ca70f-eb44-4b86-a941-cd393688d71d admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.521506 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482d76d-b4f6-463b-89a8-10fbb7dd4ffd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.521777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ca70f-eb44-4b86-a941-cd393688d71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.522196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ca70f-eb44-4b86-a941-cd393688d71d admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.523059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ca70f-eb44-4b86-a941-cd393688d71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.525146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ca70f-eb44-4b86-a941-cd393688d71d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.556475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:10.615655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 955/1912] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:10 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:10.620395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482d76d-b4f6-463b-89a8-10fbb7dd4ffd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.641683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:10.646686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 958/1913] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:10 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:10.651502 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ca70f-eb44-4b86-a941-cd393688d71d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.693076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482d76d-b4f6-463b-89a8-10fbb7dd4ffd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.717576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ca70f-eb44-4b86-a941-cd393688d71d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.722722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ddde7-b208-43cb-a872-d9722e61410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.726546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ddde7-b208-43cb-a872-d9722e61410d admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.726546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ddde7-b208-43cb-a872-d9722e61410d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.726546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ddde7-b208-43cb-a872-d9722e61410d admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.726546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ddde7-b208-43cb-a872-d9722e61410d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.726826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ddde7-b208-43cb-a872-d9722e61410d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.757310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33f96f-cc2c-4d46-9aed-f04b0b8ffcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.757310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33f96f-cc2c-4d46-9aed-f04b0b8ffcbe admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.757310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33f96f-cc2c-4d46-9aed-f04b0b8ffcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.757310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33f96f-cc2c-4d46-9aed-f04b0b8ffcbe admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.757726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33f96f-cc2c-4d46-9aed-f04b0b8ffcbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.757726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33f96f-cc2c-4d46-9aed-f04b0b8ffcbe admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.842998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 959/1914] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:10 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:10.847073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ddde7-b208-43cb-a872-d9722e61410d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.858246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ddde7-b208-43cb-a872-d9722e61410d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.865844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 956/1915] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:10 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:10.869511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe469bef-88b7-43bc-b202-dbdd2d9c4dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.870787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe469bef-88b7-43bc-b202-dbdd2d9c4dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.871140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe469bef-88b7-43bc-b202-dbdd2d9c4dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.871286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33f96f-cc2c-4d46-9aed-f04b0b8ffcbe admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.871430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe469bef-88b7-43bc-b202-dbdd2d9c4dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.872205 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe469bef-88b7-43bc-b202-dbdd2d9c4dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.873962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe469bef-88b7-43bc-b202-dbdd2d9c4dcc admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.886922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 960/1916] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:10.891096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe469bef-88b7-43bc-b202-dbdd2d9c4dcc admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.903770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33f96f-cc2c-4d46-9aed-f04b0b8ffcbe admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.910131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f3051-b0c5-4f63-9644-8edb1c8cb6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.910425 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe469bef-88b7-43bc-b202-dbdd2d9c4dcc admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.910791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f3051-b0c5-4f63-9644-8edb1c8cb6ae admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.911117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f3051-b0c5-4f63-9644-8edb1c8cb6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.911466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f3051-b0c5-4f63-9644-8edb1c8cb6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.912173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f3051-b0c5-4f63-9644-8edb1c8cb6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.913847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f3051-b0c5-4f63-9644-8edb1c8cb6ae admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.917359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77a3d8-1770-4cfa-a979-61c08e65f42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.918523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77a3d8-1770-4cfa-a979-61c08e65f42c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.918523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77a3d8-1770-4cfa-a979-61c08e65f42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.918717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77a3d8-1770-4cfa-a979-61c08e65f42c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.919462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77a3d8-1770-4cfa-a979-61c08e65f42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.921266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77a3d8-1770-4cfa-a979-61c08e65f42c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.934021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 957/1917] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:10.934021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f3051-b0c5-4f63-9644-8edb1c8cb6ae admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.939992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f3051-b0c5-4f63-9644-8edb1c8cb6ae admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.940434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 961/1918] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:10.944164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77a3d8-1770-4cfa-a979-61c08e65f42c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.950475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b00a5d-ecb1-4378-87d1-c140c3c687ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.952081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b00a5d-ecb1-4378-87d1-c140c3c687ba admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.952081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b00a5d-ecb1-4378-87d1-c140c3c687ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.952081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b00a5d-ecb1-4378-87d1-c140c3c687ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.952540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b00a5d-ecb1-4378-87d1-c140c3c687ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.953927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b00a5d-ecb1-4378-87d1-c140c3c687ba admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:10.958154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77a3d8-1770-4cfa-a979-61c08e65f42c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.965082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 958/1919] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:10.965511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37f984-6a7c-4ac6-8f51-f96a171ae05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.966408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37f984-6a7c-4ac6-8f51-f96a171ae05e admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.966948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37f984-6a7c-4ac6-8f51-f96a171ae05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.967483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37f984-6a7c-4ac6-8f51-f96a171ae05e admin admin] PATH_INFO: `/v3/projects/3b876704aecf424ab54cd836d65aadbc/users/288aee37d37e49c0a4600e708e9ac3b6/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.969279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b00a5d-ecb1-4378-87d1-c140c3c687ba admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:10.980411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b00a5d-ecb1-4378-87d1-c140c3c687ba admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:10.985216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a6ba0-66d6-4307-b6a9-d5ddeca21563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:10.985867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a6ba0-66d6-4307-b6a9-d5ddeca21563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:10.986076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37f984-6a7c-4ac6-8f51-f96a171ae05e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b876704aecf424ab54cd836d65aadbc, user_id=288aee37d37e49c0a4600e708e9ac3b6, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:10.986185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a6ba0-66d6-4307-b6a9-d5ddeca21563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:10.986387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a6ba0-66d6-4307-b6a9-d5ddeca21563 admin admin] PATH_INFO: `/v3/projects/503ced8fc9834dc5b4e6dcca6a37267d/users/8d227804e06349fead4a9a2ef5c0a72a/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:10.990289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37f984-6a7c-4ac6-8f51-f96a171ae05e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:11.002341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a6ba0-66d6-4307-b6a9-d5ddeca21563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=503ced8fc9834dc5b4e6dcca6a37267d, user_id=8d227804e06349fead4a9a2ef5c0a72a, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:11.006671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a6ba0-66d6-4307-b6a9-d5ddeca21563 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:11.089164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 959/1920] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:10 2022] PUT /identity/v3/projects/503ced8fc9834dc5b4e6dcca6a37267d/users/8d227804e06349fead4a9a2ef5c0a72a/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:11.090187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 962/1921] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:10 2022] PUT /identity/v3/projects/3b876704aecf424ab54cd836d65aadbc/users/288aee37d37e49c0a4600e708e9ac3b6/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:11.094929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c7c7a-4710-44c9-a477-5e4546df5fd4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.095214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c7c7a-4710-44c9-a477-5e4546df5fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.095486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c7c7a-4710-44c9-a477-5e4546df5fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.108526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6404092-b76f-4def-8b01-973638cddc02 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.109051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6404092-b76f-4def-8b01-973638cddc02 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.109557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6404092-b76f-4def-8b01-973638cddc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.119577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5c3c7c7a-4710-44c9-a477-5e4546df5fd4 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:11.121421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5c3c7c7a-4710-44c9-a477-5e4546df5fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:11.156099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d6404092-b76f-4def-8b01-973638cddc02 None None] MFA Rules not processed for user `8d227804e06349fead4a9a2ef5c0a72a`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:11.159061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d6404092-b76f-4def-8b01-973638cddc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:11.202288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 960/1922] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:11.214139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c7c7a-4710-44c9-a477-5e4546df5fd4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:11.215718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5c3c7c7a-4710-44c9-a477-5e4546df5fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:11.230004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:11.242783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:11.258548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 963/1923] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:11.259584 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6404092-b76f-4def-8b01-973638cddc02 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:11.270757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6404092-b76f-4def-8b01-973638cddc02 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:11.277126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3abea-4c70-4589-aa61-ae41d8a1292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:11.278000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3abea-4c70-4589-aa61-ae41d8a1292b admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.278373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3abea-4c70-4589-aa61-ae41d8a1292b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.278716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3abea-4c70-4589-aa61-ae41d8a1292b admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.279418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3abea-4c70-4589-aa61-ae41d8a1292b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:11.281517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3abea-4c70-4589-aa61-ae41d8a1292b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:11.284222 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c7c7a-4710-44c9-a477-5e4546df5fd4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:11.287865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a6275-be26-47e9-83b4-6bb821cec3c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:11.288717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a6275-be26-47e9-83b4-6bb821cec3c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.289062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a6275-be26-47e9-83b4-6bb821cec3c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.289727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a6275-be26-47e9-83b4-6bb821cec3c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.300993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a6275-be26-47e9-83b4-6bb821cec3c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:11.302513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a6275-be26-47e9-83b4-6bb821cec3c7 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:11.313105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 961/1924] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:11.362540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a6275-be26-47e9-83b4-6bb821cec3c7 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:11.364137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-063a6275-be26-47e9-83b4-6bb821cec3c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:11.369758 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 964/1925] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:11 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:11.370985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3abea-4c70-4589-aa61-ae41d8a1292b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:11.383179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a6275-be26-47e9-83b4-6bb821cec3c7 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:11.442649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3abea-4c70-4589-aa61-ae41d8a1292b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:11.452433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcaa2d-75ac-4087-bd5c-643a8b22b270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:11.453889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcaa2d-75ac-4087-bd5c-643a8b22b270 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.454832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcaa2d-75ac-4087-bd5c-643a8b22b270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.454832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcaa2d-75ac-4087-bd5c-643a8b22b270 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.456041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcaa2d-75ac-4087-bd5c-643a8b22b270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:11.459207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcaa2d-75ac-4087-bd5c-643a8b22b270 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:11.461130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:11.461908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.462280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.462653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] PATH_INFO: `/v3/users/e2c78172690e4e619e6193ffb7bdd5aa` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.466054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2c78172690e4e619e6193ffb7bdd5aa)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:11.467584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:11.564647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 965/1926] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:11 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:11.567363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcaa2d-75ac-4087-bd5c-643a8b22b270 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:11.588387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcaa2d-75ac-4087-bd5c-643a8b22b270 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:11.602816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37717a33-2d24-46ac-91e3-ba87b9435bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:11.602816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37717a33-2d24-46ac-91e3-ba87b9435bec admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.602816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37717a33-2d24-46ac-91e3-ba87b9435bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.602816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37717a33-2d24-46ac-91e3-ba87b9435bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.603151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37717a33-2d24-46ac-91e3-ba87b9435bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:11.603151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37717a33-2d24-46ac-91e3-ba87b9435bec admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:11.612616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 966/1927] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:11.617393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37717a33-2d24-46ac-91e3-ba87b9435bec admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:11.630693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37717a33-2d24-46ac-91e3-ba87b9435bec admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:11.634969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2c78172690e4e619e6193ffb7bdd5aa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:11.637518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c07df9-e990-4980-88ad-bfb48ca8d0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:11.638129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c07df9-e990-4980-88ad-bfb48ca8d0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.638405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c07df9-e990-4980-88ad-bfb48ca8d0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.638672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c07df9-e990-4980-88ad-bfb48ca8d0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.639315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c07df9-e990-4980-88ad-bfb48ca8d0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:11.641672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c07df9-e990-4980-88ad-bfb48ca8d0e9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:11.660234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 967/1928] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:11.661257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c07df9-e990-4980-88ad-bfb48ca8d0e9 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:11.673658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c07df9-e990-4980-88ad-bfb48ca8d0e9 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:11.678545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4568901-c03b-456f-98d1-93f62320adf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:11.679195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4568901-c03b-456f-98d1-93f62320adf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.679498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4568901-c03b-456f-98d1-93f62320adf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.679765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4568901-c03b-456f-98d1-93f62320adf4 admin admin] PATH_INFO: `/v3/projects/296615f4b50e4903bd90bfe2ce22be8d/users/5174cb621c4e4f8692ebd4349edca2c7/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.700119 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4568901-c03b-456f-98d1-93f62320adf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296615f4b50e4903bd90bfe2ce22be8d, user_id=5174cb621c4e4f8692ebd4349edca2c7, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:11.702878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4568901-c03b-456f-98d1-93f62320adf4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:11.710677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2c78172690e4e619e6193ffb7bdd5aa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:11.721993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2c78172690e4e619e6193ffb7bdd5aa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:11.722979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2c78172690e4e619e6193ffb7bdd5aa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:11.723776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-227e5605-a0c5-40dd-81be-90e3dc7fa96a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2c78172690e4e619e6193ffb7bdd5aa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:11.758908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 962/1929] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:11 2022] DELETE /identity/v3/users/e2c78172690e4e619e6193ffb7bdd5aa => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:11.764782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcccf2-174e-47e9-9866-d7d18cf3a6cd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:11.765090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcccf2-174e-47e9-9866-d7d18cf3a6cd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:11.765509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcccf2-174e-47e9-9866-d7d18cf3a6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:11.790292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-71fcccf2-174e-47e9-9866-d7d18cf3a6cd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:11.791466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-71fcccf2-174e-47e9-9866-d7d18cf3a6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:12.018128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 968/1930] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:11 2022] PUT /identity/v3/projects/296615f4b50e4903bd90bfe2ce22be8d/users/5174cb621c4e4f8692ebd4349edca2c7/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:12.018128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231b97a-7b27-4e58-b223-76cbb3ce87b4 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.018128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231b97a-7b27-4e58-b223-76cbb3ce87b4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.018128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231b97a-7b27-4e58-b223-76cbb3ce87b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.019657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:12.020133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8231b97a-7b27-4e58-b223-76cbb3ce87b4 None None] MFA Rules not processed for user `5174cb621c4e4f8692ebd4349edca2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:12.020133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8231b97a-7b27-4e58-b223-76cbb3ce87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:12.020364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 963/1931] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:12.020364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcccf2-174e-47e9-9866-d7d18cf3a6cd None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:12.020364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-71fcccf2-174e-47e9-9866-d7d18cf3a6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:12.020983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 969/1932] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:12.020983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231b97a-7b27-4e58-b223-76cbb3ce87b4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:12.020983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8231b97a-7b27-4e58-b223-76cbb3ce87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:12.020983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:12.020983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231b97a-7b27-4e58-b223-76cbb3ce87b4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:12.021323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9788cce-a3c0-4ebe-b6a4-1cc6299c14f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:12.021323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9788cce-a3c0-4ebe-b6a4-1cc6299c14f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.021323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9788cce-a3c0-4ebe-b6a4-1cc6299c14f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.021323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9788cce-a3c0-4ebe-b6a4-1cc6299c14f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.022539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9788cce-a3c0-4ebe-b6a4-1cc6299c14f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:12.022539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9788cce-a3c0-4ebe-b6a4-1cc6299c14f3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:12.025123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcccf2-174e-47e9-9866-d7d18cf3a6cd None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:12.035240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373fe4a-e0c4-406f-b3d5-1716783b630a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:12.035916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373fe4a-e0c4-406f-b3d5-1716783b630a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.036202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373fe4a-e0c4-406f-b3d5-1716783b630a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.036475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373fe4a-e0c4-406f-b3d5-1716783b630a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.049719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373fe4a-e0c4-406f-b3d5-1716783b630a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:12.050168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373fe4a-e0c4-406f-b3d5-1716783b630a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:12.067710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 964/1933] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:12.170925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 970/1934] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:11 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:12.193821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373fe4a-e0c4-406f-b3d5-1716783b630a service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:12.193821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373fe4a-e0c4-406f-b3d5-1716783b630a service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:12.292207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d26361-ccb7-473d-bb6d-7d03bc21ad72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:12.292469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d26361-ccb7-473d-bb6d-7d03bc21ad72 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.292695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d26361-ccb7-473d-bb6d-7d03bc21ad72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.292925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d26361-ccb7-473d-bb6d-7d03bc21ad72 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.293558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d26361-ccb7-473d-bb6d-7d03bc21ad72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:12.295488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d26361-ccb7-473d-bb6d-7d03bc21ad72 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:12.399390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 965/1935] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:12 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:12.440355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9788cce-a3c0-4ebe-b6a4-1cc6299c14f3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:12.440355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9788cce-a3c0-4ebe-b6a4-1cc6299c14f3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:12.440355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3e5a5-4fd9-411e-bb52-d133e0bf5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:12.440355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3e5a5-4fd9-411e-bb52-d133e0bf5e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.440753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3e5a5-4fd9-411e-bb52-d133e0bf5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.440753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3e5a5-4fd9-411e-bb52-d133e0bf5e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.440753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3e5a5-4fd9-411e-bb52-d133e0bf5e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:12.440753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3e5a5-4fd9-411e-bb52-d133e0bf5e9d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:12.448547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 971/1936] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:12.451038 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d26361-ccb7-473d-bb6d-7d03bc21ad72 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:12.468636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d26361-ccb7-473d-bb6d-7d03bc21ad72 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:12.476734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85ccff-c2d2-4a82-9d3a-e3e9858aacb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:12.481595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85ccff-c2d2-4a82-9d3a-e3e9858aacb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.481595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85ccff-c2d2-4a82-9d3a-e3e9858aacb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.481595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85ccff-c2d2-4a82-9d3a-e3e9858aacb3 admin admin] PATH_INFO: `/v3/projects/8a0a7f2c8a8e42efbfa8fd7f42e3157c/users/18197194fa60410f9050f877e314b0d5/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.503267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85ccff-c2d2-4a82-9d3a-e3e9858aacb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0a7f2c8a8e42efbfa8fd7f42e3157c, user_id=18197194fa60410f9050f877e314b0d5, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:12.510122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85ccff-c2d2-4a82-9d3a-e3e9858aacb3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:12.590922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 966/1937] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:12 2022] PUT /identity/v3/projects/8a0a7f2c8a8e42efbfa8fd7f42e3157c/users/18197194fa60410f9050f877e314b0d5/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:12.594848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3e5a5-4fd9-411e-bb52-d133e0bf5e9d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:12.605641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3e5a5-4fd9-411e-bb52-d133e0bf5e9d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:12.669985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45de9cd-9867-440c-8a20-6b4ec9063604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:12.669985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45de9cd-9867-440c-8a20-6b4ec9063604 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.669985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45de9cd-9867-440c-8a20-6b4ec9063604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.669985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45de9cd-9867-440c-8a20-6b4ec9063604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.670440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45de9cd-9867-440c-8a20-6b4ec9063604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:12.671730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45de9cd-9867-440c-8a20-6b4ec9063604 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:12.681649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 972/1938] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:12.685865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85ccff-c2d2-4a82-9d3a-e3e9858aacb3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:12.696356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85ccff-c2d2-4a82-9d3a-e3e9858aacb3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:12.701927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f7244-aadf-4b3c-a928-af64584a2a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:12.702646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f7244-aadf-4b3c-a928-af64584a2a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.703037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f7244-aadf-4b3c-a928-af64584a2a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.703369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f7244-aadf-4b3c-a928-af64584a2a0e admin admin] PATH_INFO: `/v3/projects/8a0a7f2c8a8e42efbfa8fd7f42e3157c/users/18197194fa60410f9050f877e314b0d5/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.705602 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f7244-aadf-4b3c-a928-af64584a2a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0a7f2c8a8e42efbfa8fd7f42e3157c, user_id=18197194fa60410f9050f877e314b0d5, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:12.712063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f7244-aadf-4b3c-a928-af64584a2a0e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:12.722921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 967/1939] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:12 2022] PUT /identity/v3/projects/8a0a7f2c8a8e42efbfa8fd7f42e3157c/users/18197194fa60410f9050f877e314b0d5/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:12.738930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb9db-933d-4ac0-9a40-ca657cee71e6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.738930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb9db-933d-4ac0-9a40-ca657cee71e6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.739409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb9db-933d-4ac0-9a40-ca657cee71e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.775029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-79dbb9db-933d-4ac0-9a40-ca657cee71e6 None None] MFA Rules not processed for user `18197194fa60410f9050f877e314b0d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:12.776858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-79dbb9db-933d-4ac0-9a40-ca657cee71e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:12.826610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f7244-aadf-4b3c-a928-af64584a2a0e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:12.829572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9d5f7244-aadf-4b3c-a928-af64584a2a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:12.923519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:12.923995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 973/1940] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:12.928322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ae6b2-b365-4044-a671-505873ac3806 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.928641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ae6b2-b365-4044-a671-505873ac3806 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.928915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ae6b2-b365-4044-a671-505873ac3806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.944443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f7244-aadf-4b3c-a928-af64584a2a0e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:12.952291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:12.959334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:12.960522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:12.960883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] PATH_INFO: `/v3/projects/08c7180c708c4b2cb4c42cb40596503e` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:12.962627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c7180c708c4b2cb4c42cb40596503e)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:12.967688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e80ae6b2-b365-4044-a671-505873ac3806 None None] MFA Rules not processed for user `288aee37d37e49c0a4600e708e9ac3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:12.969026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:12.969337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e80ae6b2-b365-4044-a671-505873ac3806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:13.033938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:13.056863 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 974/1941] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:13.057741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ae6b2-b365-4044-a671-505873ac3806 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:13.058999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e80ae6b2-b365-4044-a671-505873ac3806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:13.091640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c7180c708c4b2cb4c42cb40596503e', 'request_id': 'req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:13.100870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c7180c708c4b2cb4c42cb40596503e', 'request_id': 'req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:13.121435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ae6b2-b365-4044-a671-505873ac3806 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:13.172086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de93fee-dca6-44d7-9bd9-431e7f529843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:13.172722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de93fee-dca6-44d7-9bd9-431e7f529843 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.172996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de93fee-dca6-44d7-9bd9-431e7f529843 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.173259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de93fee-dca6-44d7-9bd9-431e7f529843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.207474 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c7180c708c4b2cb4c42cb40596503e', 'request_id': 'req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:13.219954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] The token cache is being invalidate because project 08c7180c708c4b2cb4c42cb40596503e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:13.220241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:13.220920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:13.222431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 968/1942] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:12 2022] DELETE /identity/v3/projects/08c7180c708c4b2cb4c42cb40596503e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:13.225412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:13.229096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:13.235670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de93fee-dca6-44d7-9bd9-431e7f529843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:13.237181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de93fee-dca6-44d7-9bd9-431e7f529843 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:13.246624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 975/1943] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:13.247842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc261b-ebaa-4e56-b57e-6bb44ceb3f08 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.248126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc261b-ebaa-4e56-b57e-6bb44ceb3f08 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.248397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc261b-ebaa-4e56-b57e-6bb44ceb3f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.268596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4ccc261b-ebaa-4e56-b57e-6bb44ceb3f08 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:13.269813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4ccc261b-ebaa-4e56-b57e-6bb44ceb3f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:13.301590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712662e7-89ad-4e20-b4c3-b5f90aad5b7f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:13.307266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d15705-f419-4203-93b0-66a7c637651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:13.308050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d15705-f419-4203-93b0-66a7c637651f admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.308496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d15705-f419-4203-93b0-66a7c637651f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.308977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d15705-f419-4203-93b0-66a7c637651f admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.321878 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d15705-f419-4203-93b0-66a7c637651f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:13.321878 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d15705-f419-4203-93b0-66a7c637651f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:13.356906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 976/1944] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:13.356906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc261b-ebaa-4e56-b57e-6bb44ceb3f08 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:13.356906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4ccc261b-ebaa-4e56-b57e-6bb44ceb3f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:13.363321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc261b-ebaa-4e56-b57e-6bb44ceb3f08 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:13.367470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058eedf8-e7a2-4126-b015-aae776db8c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:13.368188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058eedf8-e7a2-4126-b015-aae776db8c2e service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.368590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058eedf8-e7a2-4126-b015-aae776db8c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.368926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058eedf8-e7a2-4126-b015-aae776db8c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.370865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-058eedf8-e7a2-4126-b015-aae776db8c2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:13.415520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 969/1945] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:13 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:13.416950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe8510-7417-4674-ad9b-f2205e82606f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.417340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe8510-7417-4674-ad9b-f2205e82606f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.417532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe8510-7417-4674-ad9b-f2205e82606f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.426639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:13.449464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1dfe8510-7417-4674-ad9b-f2205e82606f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:13.450917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1dfe8510-7417-4674-ad9b-f2205e82606f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:13.452337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058eedf8-e7a2-4126-b015-aae776db8c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:13.453834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058eedf8-e7a2-4126-b015-aae776db8c2e service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:13.465658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 977/1946] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:13.471476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058eedf8-e7a2-4126-b015-aae776db8c2e service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:13.480076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058eedf8-e7a2-4126-b015-aae776db8c2e service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:13.517731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:13.542352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 970/1947] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:13.546832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe8510-7417-4674-ad9b-f2205e82606f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:13.559414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe8510-7417-4674-ad9b-f2205e82606f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:13.566491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03becbc4-cb33-4dd0-b4d0-36db8617828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:13.567242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03becbc4-cb33-4dd0-b4d0-36db8617828b admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.567627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03becbc4-cb33-4dd0-b4d0-36db8617828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.568025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03becbc4-cb33-4dd0-b4d0-36db8617828b admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.569295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03becbc4-cb33-4dd0-b4d0-36db8617828b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:13.569600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03becbc4-cb33-4dd0-b4d0-36db8617828b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:13.571863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03becbc4-cb33-4dd0-b4d0-36db8617828b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:13.588686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 971/1948] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:10:13.593177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03becbc4-cb33-4dd0-b4d0-36db8617828b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:13.603619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03becbc4-cb33-4dd0-b4d0-36db8617828b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:13.608757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7064c-6fd2-499d-89b8-7a89049f2c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:13.608888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e4e4e-ec4a-40c6-8a10-3ecbf994bc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:13.609293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7064c-6fd2-499d-89b8-7a89049f2c99 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.609567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7064c-6fd2-499d-89b8-7a89049f2c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.609772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e4e4e-ec4a-40c6-8a10-3ecbf994bc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.609877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7064c-6fd2-499d-89b8-7a89049f2c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.610131 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e4e4e-ec4a-40c6-8a10-3ecbf994bc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.610429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7064c-6fd2-499d-89b8-7a89049f2c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:13.610537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e4e4e-ec4a-40c6-8a10-3ecbf994bc54 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.611187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e4e4e-ec4a-40c6-8a10-3ecbf994bc54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:13.612481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7064c-6fd2-499d-89b8-7a89049f2c99 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:13.613560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e4e4e-ec4a-40c6-8a10-3ecbf994bc54 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:13.742642 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 972/1949] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:13 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:13.754303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7064c-6fd2-499d-89b8-7a89049f2c99 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:13.771442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 978/1950] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:13 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:13.775805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e4e4e-ec4a-40c6-8a10-3ecbf994bc54 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:13.794986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e4e4e-ec4a-40c6-8a10-3ecbf994bc54 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:13.873133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7064c-6fd2-499d-89b8-7a89049f2c99 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:13.878001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54269533-f04d-4291-80f4-c4b188a98d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:13.879354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54269533-f04d-4291-80f4-c4b188a98d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.879354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54269533-f04d-4291-80f4-c4b188a98d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.879354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54269533-f04d-4291-80f4-c4b188a98d2b admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.879947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54269533-f04d-4291-80f4-c4b188a98d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:13.883230 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54269533-f04d-4291-80f4-c4b188a98d2b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:13.908320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594073ee-dfbd-45a5-abc1-4e036aa1adcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:13.908320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594073ee-dfbd-45a5-abc1-4e036aa1adcd admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.908320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594073ee-dfbd-45a5-abc1-4e036aa1adcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.908320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594073ee-dfbd-45a5-abc1-4e036aa1adcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.908994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594073ee-dfbd-45a5-abc1-4e036aa1adcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:13.912469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594073ee-dfbd-45a5-abc1-4e036aa1adcd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:13.923897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 979/1951] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:13.928145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594073ee-dfbd-45a5-abc1-4e036aa1adcd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:13.940808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594073ee-dfbd-45a5-abc1-4e036aa1adcd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:13.945972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75234d69-8183-4eec-8072-2f9ae1ddd586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:13.946863 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75234d69-8183-4eec-8072-2f9ae1ddd586 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:13.947292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75234d69-8183-4eec-8072-2f9ae1ddd586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:13.947716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75234d69-8183-4eec-8072-2f9ae1ddd586 admin admin] PATH_INFO: `/v3/projects/4d3053f55feb4a8abf1234688d448f0d/users/7fdc81dc9eb14461bcf544d2a417dc9f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:13.964854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:13.971999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75234d69-8183-4eec-8072-2f9ae1ddd586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3053f55feb4a8abf1234688d448f0d, user_id=7fdc81dc9eb14461bcf544d2a417dc9f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:13.974695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75234d69-8183-4eec-8072-2f9ae1ddd586 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:13.988418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 973/1952] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:13 2022] POST /identity/v3/users => generated 287 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:13.994207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54269533-f04d-4291-80f4-c4b188a98d2b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:14.010966 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54269533-f04d-4291-80f4-c4b188a98d2b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:14.017623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a16f6-bce6-4c92-b4d7-c5bca6544f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:14.018434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a16f6-bce6-4c92-b4d7-c5bca6544f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:14.018546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a16f6-bce6-4c92-b4d7-c5bca6544f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:14.018756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a16f6-bce6-4c92-b4d7-c5bca6544f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:14.019404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a16f6-bce6-4c92-b4d7-c5bca6544f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:14.021084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a16f6-bce6-4c92-b4d7-c5bca6544f93 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:14.032788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 974/1953] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:14.036382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a16f6-bce6-4c92-b4d7-c5bca6544f93 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:14.051337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a16f6-bce6-4c92-b4d7-c5bca6544f93 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:14.056339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae623d2-c65a-4a6b-b4df-90a0bba2b8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:14.057935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae623d2-c65a-4a6b-b4df-90a0bba2b8fe admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:14.058084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae623d2-c65a-4a6b-b4df-90a0bba2b8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:14.059850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae623d2-c65a-4a6b-b4df-90a0bba2b8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:14.061710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae623d2-c65a-4a6b-b4df-90a0bba2b8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:14.063370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae623d2-c65a-4a6b-b4df-90a0bba2b8fe admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:14.075963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 980/1954] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:13 2022] PUT /identity/v3/projects/4d3053f55feb4a8abf1234688d448f0d/users/7fdc81dc9eb14461bcf544d2a417dc9f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:14.080313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75234d69-8183-4eec-8072-2f9ae1ddd586 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:14.081350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 975/1955] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:14.089417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae623d2-c65a-4a6b-b4df-90a0bba2b8fe admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:14.103640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae623d2-c65a-4a6b-b4df-90a0bba2b8fe admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:14.176217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fb052-cc8d-4236-9a8e-77ee3a06bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:14.177288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fb052-cc8d-4236-9a8e-77ee3a06bbba admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:14.177807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fb052-cc8d-4236-9a8e-77ee3a06bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:14.178210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fb052-cc8d-4236-9a8e-77ee3a06bbba admin admin] PATH_INFO: `/v3/projects/64273458329d44edb0e2904b4011b591/users/b7d7f6721a3a40f380578391949c443b/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:14.198628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fb052-cc8d-4236-9a8e-77ee3a06bbba admin admin] RBAC: Authorizing `identity:create_grant(project_id=64273458329d44edb0e2904b4011b591, user_id=b7d7f6721a3a40f380578391949c443b, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:14.200257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75234d69-8183-4eec-8072-2f9ae1ddd586 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:14.204313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fb052-cc8d-4236-9a8e-77ee3a06bbba admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:14.207790 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305718c-7a4c-4a9f-a989-a539f05bc929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:14.208535 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305718c-7a4c-4a9f-a989-a539f05bc929 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:14.209016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305718c-7a4c-4a9f-a989-a539f05bc929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:14.209291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305718c-7a4c-4a9f-a989-a539f05bc929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:14.210373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305718c-7a4c-4a9f-a989-a539f05bc929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:14.211746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305718c-7a4c-4a9f-a989-a539f05bc929 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:14.225897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 981/1956] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:14.233234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305718c-7a4c-4a9f-a989-a539f05bc929 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:14.242071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305718c-7a4c-4a9f-a989-a539f05bc929 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:14.247548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d893e9-c32b-4c79-bb57-7e00f0a6af5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:14.248342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d893e9-c32b-4c79-bb57-7e00f0a6af5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:14.248720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d893e9-c32b-4c79-bb57-7e00f0a6af5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:14.249169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d893e9-c32b-4c79-bb57-7e00f0a6af5d admin admin] PATH_INFO: `/v3/projects/4d3053f55feb4a8abf1234688d448f0d/users/7fdc81dc9eb14461bcf544d2a417dc9f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:14.251229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d893e9-c32b-4c79-bb57-7e00f0a6af5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3053f55feb4a8abf1234688d448f0d, user_id=7fdc81dc9eb14461bcf544d2a417dc9f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:14.254251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d893e9-c32b-4c79-bb57-7e00f0a6af5d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:14.279030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 982/1957] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:14 2022] PUT /identity/v3/projects/4d3053f55feb4a8abf1234688d448f0d/users/7fdc81dc9eb14461bcf544d2a417dc9f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:14.294824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:14.301718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 976/1958] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:14 2022] PUT /identity/v3/projects/64273458329d44edb0e2904b4011b591/users/b7d7f6721a3a40f380578391949c443b/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:14.325969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828116b-75c9-423d-b60e-827bf8bf26db None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:14.326633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828116b-75c9-423d-b60e-827bf8bf26db None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:14.326941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828116b-75c9-423d-b60e-827bf8bf26db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:14.366749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7828116b-75c9-423d-b60e-827bf8bf26db None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:14.367944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7828116b-75c9-423d-b60e-827bf8bf26db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:14.437751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 983/1959] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:14.461342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fb052-cc8d-4236-9a8e-77ee3a06bbba admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:14.463699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-315fb052-cc8d-4236-9a8e-77ee3a06bbba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:14.552954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fb052-cc8d-4236-9a8e-77ee3a06bbba admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:14.558213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c3d42-ab27-4523-8be1-33ce92f2f467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:14.558842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c3d42-ab27-4523-8be1-33ce92f2f467 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:14.560905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c3d42-ab27-4523-8be1-33ce92f2f467 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:14.561203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c3d42-ab27-4523-8be1-33ce92f2f467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:14.570799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c3d42-ab27-4523-8be1-33ce92f2f467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:14.572247 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c3d42-ab27-4523-8be1-33ce92f2f467 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:14.582311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 977/1960] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:14.656525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828116b-75c9-423d-b60e-827bf8bf26db None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:14.658067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7828116b-75c9-423d-b60e-827bf8bf26db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:14.671485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828116b-75c9-423d-b60e-827bf8bf26db None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:14.679881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:14.679881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:14.679881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:14.679881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] PATH_INFO: `/v3/users/ec40d2360ab2417bb2d50414575d59d0` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:14.680257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec40d2360ab2417bb2d50414575d59d0)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:14.680679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:14.947531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec40d2360ab2417bb2d50414575d59d0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.030073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec40d2360ab2417bb2d50414575d59d0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.038595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec40d2360ab2417bb2d50414575d59d0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.039352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec40d2360ab2417bb2d50414575d59d0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.039945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-5092613f-fac4-4f21-b58e-fb5fd4c12f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec40d2360ab2417bb2d50414575d59d0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.044518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:15.066105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 984/1961] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:14 2022] DELETE /identity/v3/users/ec40d2360ab2417bb2d50414575d59d0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:15.588188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c3d42-ab27-4523-8be1-33ce92f2f467 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:15.610752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c3d42-ab27-4523-8be1-33ce92f2f467 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:15.688554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:15.689464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:15.689879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:15.690159 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] PATH_INFO: `/v3/projects/7b16538b6545458aa54f1edd7f5c5737` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:15.691291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b16538b6545458aa54f1edd7f5c5737)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:15.693267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:15.811599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b16538b6545458aa54f1edd7f5c5737', 'request_id': 'req-9b1436c2-062d-4957-b11a-4420df63b7c0'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.817089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b16538b6545458aa54f1edd7f5c5737', 'request_id': 'req-9b1436c2-062d-4957-b11a-4420df63b7c0'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.887129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b16538b6545458aa54f1edd7f5c5737', 'request_id': 'req-9b1436c2-062d-4957-b11a-4420df63b7c0'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.899508 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] The token cache is being invalidate because project 7b16538b6545458aa54f1edd7f5c5737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:15.899778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.901655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b1436c2-062d-4957-b11a-4420df63b7c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:15.904570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 978/1962] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:15 2022] DELETE /identity/v3/projects/7b16538b6545458aa54f1edd7f5c5737 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:15.909364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82d4b-1609-4097-a0fa-10b27e79c61d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:15.909778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82d4b-1609-4097-a0fa-10b27e79c61d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:15.909933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82d4b-1609-4097-a0fa-10b27e79c61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:15.935344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2cf82d4b-1609-4097-a0fa-10b27e79c61d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:15.936677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2cf82d4b-1609-4097-a0fa-10b27e79c61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:16.011691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 985/1963] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:16.019947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182054f3-dd50-45be-9071-e20723dce030 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.020718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182054f3-dd50-45be-9071-e20723dce030 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.021070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182054f3-dd50-45be-9071-e20723dce030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.043750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-182054f3-dd50-45be-9071-e20723dce030 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:16.046407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-182054f3-dd50-45be-9071-e20723dce030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:16.054561 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 979/1964] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:16.063485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82d4b-1609-4097-a0fa-10b27e79c61d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:16.074311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82d4b-1609-4097-a0fa-10b27e79c61d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:16.075686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667869fd-0050-4058-a0fd-ed12762a11b0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.075974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667869fd-0050-4058-a0fd-ed12762a11b0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.076244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667869fd-0050-4058-a0fd-ed12762a11b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.079943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e072f6-4608-4457-9bad-eb16d339eb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:16.081071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e072f6-4608-4457-9bad-eb16d339eb3c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.081340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e072f6-4608-4457-9bad-eb16d339eb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.081754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e072f6-4608-4457-9bad-eb16d339eb3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.082257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e072f6-4608-4457-9bad-eb16d339eb3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:16.082552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e072f6-4608-4457-9bad-eb16d339eb3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:16.084180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e072f6-4608-4457-9bad-eb16d339eb3c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:16.103259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 986/1965] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:16.105003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e072f6-4608-4457-9bad-eb16d339eb3c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:16.117264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e072f6-4608-4457-9bad-eb16d339eb3c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:16.126658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75351a6a-84e2-4a65-8312-cb721147d847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:16.126658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75351a6a-84e2-4a65-8312-cb721147d847 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.126658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75351a6a-84e2-4a65-8312-cb721147d847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.126658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75351a6a-84e2-4a65-8312-cb721147d847 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.129375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75351a6a-84e2-4a65-8312-cb721147d847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:16.129467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75351a6a-84e2-4a65-8312-cb721147d847 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:16.196603 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-667869fd-0050-4058-a0fd-ed12762a11b0 None None] MFA Rules not processed for user `7fdc81dc9eb14461bcf544d2a417dc9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:16.198351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-667869fd-0050-4058-a0fd-ed12762a11b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:16.213072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:16.216980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 987/1966] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:16 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:16.221159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75351a6a-84e2-4a65-8312-cb721147d847 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:16.286246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 980/1967] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:16.296955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667869fd-0050-4058-a0fd-ed12762a11b0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:16.299054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-667869fd-0050-4058-a0fd-ed12762a11b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:16.314398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75351a6a-84e2-4a65-8312-cb721147d847 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:16.323387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a11df-4e81-4fa8-8ba4-6597b0852fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:16.324932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a11df-4e81-4fa8-8ba4-6597b0852fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.325437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a11df-4e81-4fa8-8ba4-6597b0852fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.326195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a11df-4e81-4fa8-8ba4-6597b0852fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.327194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a11df-4e81-4fa8-8ba4-6597b0852fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:16.331257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a11df-4e81-4fa8-8ba4-6597b0852fa2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:16.390060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667869fd-0050-4058-a0fd-ed12762a11b0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:16.444934 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 988/1968] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:16 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:16.449908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a11df-4e81-4fa8-8ba4-6597b0852fa2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:16.462460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a11df-4e81-4fa8-8ba4-6597b0852fa2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:16.472879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2c02a-b0e5-4a0d-adad-7dfeb3fb719c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:16.478531 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf2ecf-6e38-4456-a6b0-222389976082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:16.478674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2c02a-b0e5-4a0d-adad-7dfeb3fb719c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.478674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2c02a-b0e5-4a0d-adad-7dfeb3fb719c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.478674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2c02a-b0e5-4a0d-adad-7dfeb3fb719c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.478674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2c02a-b0e5-4a0d-adad-7dfeb3fb719c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:16.479068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf2ecf-6e38-4456-a6b0-222389976082 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.479068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf2ecf-6e38-4456-a6b0-222389976082 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.479068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf2ecf-6e38-4456-a6b0-222389976082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.479068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-83cf2ecf-6e38-4456-a6b0-222389976082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:16.479892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2c02a-b0e5-4a0d-adad-7dfeb3fb719c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:16.508157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 989/1969] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:16.511295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2c02a-b0e5-4a0d-adad-7dfeb3fb719c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:16.532375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2c02a-b0e5-4a0d-adad-7dfeb3fb719c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:16.544487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47f77f-2bf2-4a47-9845-6ba0d85ab0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:16.545654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47f77f-2bf2-4a47-9845-6ba0d85ab0ae admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.546096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47f77f-2bf2-4a47-9845-6ba0d85ab0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.546524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47f77f-2bf2-4a47-9845-6ba0d85ab0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.547328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47f77f-2bf2-4a47-9845-6ba0d85ab0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:16.549581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47f77f-2bf2-4a47-9845-6ba0d85ab0ae admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:16.562426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 990/1970] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:16.567504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47f77f-2bf2-4a47-9845-6ba0d85ab0ae admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:16.577041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf2ecf-6e38-4456-a6b0-222389976082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:16.579599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf2ecf-6e38-4456-a6b0-222389976082 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:16.592977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 981/1971] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:16.601554 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47f77f-2bf2-4a47-9845-6ba0d85ab0ae admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:16.612640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c829f-ae78-4d64-9019-11c4804acd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:16.613483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c829f-ae78-4d64-9019-11c4804acd76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.613819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c829f-ae78-4d64-9019-11c4804acd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.614096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c829f-ae78-4d64-9019-11c4804acd76 admin admin] PATH_INFO: `/v3/projects/aaf010b6bea84e77a58a124a7592ccca/users/7bc24f38528d4cf6b60734126def9447/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.642410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c829f-ae78-4d64-9019-11c4804acd76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf010b6bea84e77a58a124a7592ccca, user_id=7bc24f38528d4cf6b60734126def9447, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:16.642410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c829f-ae78-4d64-9019-11c4804acd76 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:16.644295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:16.722750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 991/1972] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:16 2022] PUT /identity/v3/projects/aaf010b6bea84e77a58a124a7592ccca/users/7bc24f38528d4cf6b60734126def9447/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:16.733146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ca01e-1c32-40fe-a78b-d7869a34b10c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.733426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ca01e-1c32-40fe-a78b-d7869a34b10c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.733681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ca01e-1c32-40fe-a78b-d7869a34b10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.764087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2c2ca01e-1c32-40fe-a78b-d7869a34b10c None None] MFA Rules not processed for user `7bc24f38528d4cf6b60734126def9447`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:16.765320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2c2ca01e-1c32-40fe-a78b-d7869a34b10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:16.832348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:16.851258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c829f-ae78-4d64-9019-11c4804acd76 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:16.855822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 982/1973] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:16.948766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c829f-ae78-4d64-9019-11c4804acd76 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:16.954616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42208fb3-f4e5-4552-b640-d7546cc81707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:16.955474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42208fb3-f4e5-4552-b640-d7546cc81707 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:16.955957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42208fb3-f4e5-4552-b640-d7546cc81707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:16.961208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42208fb3-f4e5-4552-b640-d7546cc81707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:16.961208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42208fb3-f4e5-4552-b640-d7546cc81707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:16.961208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42208fb3-f4e5-4552-b640-d7546cc81707 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:17.074850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 992/1974] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:16 2022] POST /identity/v3/projects => generated 367 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:17.078722 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ca01e-1c32-40fe-a78b-d7869a34b10c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:17.090224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ca01e-1c32-40fe-a78b-d7869a34b10c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:17.149370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a0592-efc2-418d-82ad-d06e127ce060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:17.150033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a0592-efc2-418d-82ad-d06e127ce060 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:17.150307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a0592-efc2-418d-82ad-d06e127ce060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:17.150573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a0592-efc2-418d-82ad-d06e127ce060 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:17.151830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a0592-efc2-418d-82ad-d06e127ce060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:17.153815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a0592-efc2-418d-82ad-d06e127ce060 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:17.248276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 983/1975] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:17 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:17.252747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42208fb3-f4e5-4552-b640-d7546cc81707 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:17.263886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42208fb3-f4e5-4552-b640-d7546cc81707 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:17.268875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db27c5-be54-45d5-a153-258960995a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:17.269621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db27c5-be54-45d5-a153-258960995a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:17.270227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db27c5-be54-45d5-a153-258960995a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:17.270691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db27c5-be54-45d5-a153-258960995a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:17.271774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db27c5-be54-45d5-a153-258960995a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:17.273729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db27c5-be54-45d5-a153-258960995a0f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:17.295708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 993/1976] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:17.309624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a0592-efc2-418d-82ad-d06e127ce060 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:17.330798 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a0592-efc2-418d-82ad-d06e127ce060 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:17.349145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85134f3d-8c9e-4ed5-8b01-2bacf9fb4903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:17.349145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85134f3d-8c9e-4ed5-8b01-2bacf9fb4903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:17.356964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85134f3d-8c9e-4ed5-8b01-2bacf9fb4903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:17.356964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85134f3d-8c9e-4ed5-8b01-2bacf9fb4903 admin admin] PATH_INFO: `/v3/projects/93637ab846764fe196da20c2c274d93f/users/d3ab9d719f05495b8ce0b0dc080838b4/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:17.406287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85134f3d-8c9e-4ed5-8b01-2bacf9fb4903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93637ab846764fe196da20c2c274d93f, user_id=d3ab9d719f05495b8ce0b0dc080838b4, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:17.415615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85134f3d-8c9e-4ed5-8b01-2bacf9fb4903 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:17.506017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 984/1977] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:17 2022] PUT /identity/v3/projects/93637ab846764fe196da20c2c274d93f/users/d3ab9d719f05495b8ce0b0dc080838b4/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:17.517528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db27c5-be54-45d5-a153-258960995a0f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:17.537278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db27c5-be54-45d5-a153-258960995a0f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:17.612706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39082cf3-64cf-4c0b-87df-1a89e5724025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:17.613307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39082cf3-64cf-4c0b-87df-1a89e5724025 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:17.613585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39082cf3-64cf-4c0b-87df-1a89e5724025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:17.613851 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39082cf3-64cf-4c0b-87df-1a89e5724025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:17.614485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39082cf3-64cf-4c0b-87df-1a89e5724025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:17.616104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39082cf3-64cf-4c0b-87df-1a89e5724025 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:17.625967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 994/1978] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:17.630239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85134f3d-8c9e-4ed5-8b01-2bacf9fb4903 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:17.640969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85134f3d-8c9e-4ed5-8b01-2bacf9fb4903 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:17.645584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9791c-86ef-457a-be31-14eca8eb56e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:17.646186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9791c-86ef-457a-be31-14eca8eb56e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:17.646463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9791c-86ef-457a-be31-14eca8eb56e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:17.646726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9791c-86ef-457a-be31-14eca8eb56e8 admin admin] PATH_INFO: `/v3/projects/93637ab846764fe196da20c2c274d93f/users/d3ab9d719f05495b8ce0b0dc080838b4/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:17.648659 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9791c-86ef-457a-be31-14eca8eb56e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93637ab846764fe196da20c2c274d93f, user_id=d3ab9d719f05495b8ce0b0dc080838b4, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:17.651927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9791c-86ef-457a-be31-14eca8eb56e8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:17.662411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 985/1979] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:17 2022] PUT /identity/v3/projects/93637ab846764fe196da20c2c274d93f/users/d3ab9d719f05495b8ce0b0dc080838b4/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:17.673922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17fdf3-4e0e-439e-bc75-4d7a69bc0c6b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:17.674248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17fdf3-4e0e-439e-bc75-4d7a69bc0c6b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:17.674551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17fdf3-4e0e-439e-bc75-4d7a69bc0c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:17.703473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-5f17fdf3-4e0e-439e-bc75-4d7a69bc0c6b None None] MFA Rules not processed for user `d3ab9d719f05495b8ce0b0dc080838b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:17.704871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5f17fdf3-4e0e-439e-bc75-4d7a69bc0c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:17.758327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:17.777818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 995/1980] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:17.789507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9791c-86ef-457a-be31-14eca8eb56e8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:17.849913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9791c-86ef-457a-be31-14eca8eb56e8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:17.853731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e713000-1a9f-4cf7-aeba-e7744624ccd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:17.854496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e713000-1a9f-4cf7-aeba-e7744624ccd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:17.854768 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e713000-1a9f-4cf7-aeba-e7744624ccd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:17.855033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e713000-1a9f-4cf7-aeba-e7744624ccd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:17.865633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e713000-1a9f-4cf7-aeba-e7744624ccd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:17.867065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e713000-1a9f-4cf7-aeba-e7744624ccd3 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:17.876866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 986/1981] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:18.008127 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17fdf3-4e0e-439e-bc75-4d7a69bc0c6b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:18.020555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17fdf3-4e0e-439e-bc75-4d7a69bc0c6b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:18.025324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f27ec-620a-435d-8101-f495f6b63a4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:18.025985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f27ec-620a-435d-8101-f495f6b63a4b service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:18.026287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f27ec-620a-435d-8101-f495f6b63a4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:18.026550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f27ec-620a-435d-8101-f495f6b63a4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:18.093734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e713000-1a9f-4cf7-aeba-e7744624ccd3 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:18.094936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3e713000-1a9f-4cf7-aeba-e7744624ccd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:18.103414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:18.106220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f27ec-620a-435d-8101-f495f6b63a4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:18.107796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f27ec-620a-435d-8101-f495f6b63a4b service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:18.117906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 996/1982] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:18.188108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e713000-1a9f-4cf7-aeba-e7744624ccd3 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:18.211608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:18.211815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:18.212191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:18.212550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] PATH_INFO: `/v3/users/8d227804e06349fead4a9a2ef5c0a72a` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:18.221583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d227804e06349fead4a9a2ef5c0a72a)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:18.223572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:18.477363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d227804e06349fead4a9a2ef5c0a72a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:18.509321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f27ec-620a-435d-8101-f495f6b63a4b service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:18.511539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f48f27ec-620a-435d-8101-f495f6b63a4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:18.529582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f27ec-620a-435d-8101-f495f6b63a4b service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:18.534683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ffced-83a9-4b58-9f34-c76b3731cc8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:18.535432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ffced-83a9-4b58-9f34-c76b3731cc8b service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:18.535709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ffced-83a9-4b58-9f34-c76b3731cc8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:18.536113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ffced-83a9-4b58-9f34-c76b3731cc8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:18.567206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d227804e06349fead4a9a2ef5c0a72a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:18.576270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d227804e06349fead4a9a2ef5c0a72a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:18.579833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d227804e06349fead4a9a2ef5c0a72a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:18.579833 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c422fbf3-e237-4120-afc8-52c81a57d147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d227804e06349fead4a9a2ef5c0a72a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:18.598604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 987/1983] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:18 2022] DELETE /identity/v3/users/8d227804e06349fead4a9a2ef5c0a72a => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:18.603517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e126ca-c156-469a-977f-516ae9b153b4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:18.603794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e126ca-c156-469a-977f-516ae9b153b4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:18.603992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e126ca-c156-469a-977f-516ae9b153b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:18.619507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ffced-83a9-4b58-9f34-c76b3731cc8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:18.621334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ffced-83a9-4b58-9f34-c76b3731cc8b service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:18.633376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-81e126ca-c156-469a-977f-516ae9b153b4 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:18.637035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 997/1984] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:18.644865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-81e126ca-c156-469a-977f-516ae9b153b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:18.766366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 988/1985] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:18.767305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ffced-83a9-4b58-9f34-c76b3731cc8b service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:18.768884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5d3ffced-83a9-4b58-9f34-c76b3731cc8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:18.900142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ffced-83a9-4b58-9f34-c76b3731cc8b service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:18.969723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:18.988014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97090b4e-cf34-4dde-87a9-d04e8e49695d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:18.988784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97090b4e-cf34-4dde-87a9-d04e8e49695d service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:18.989121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97090b4e-cf34-4dde-87a9-d04e8e49695d service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:18.989450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97090b4e-cf34-4dde-87a9-d04e8e49695d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:19.013641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97090b4e-cf34-4dde-87a9-d04e8e49695d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:19.015710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97090b4e-cf34-4dde-87a9-d04e8e49695d service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:19.035556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 998/1986] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:19.598581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e126ca-c156-469a-977f-516ae9b153b4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:19.601732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-81e126ca-c156-469a-977f-516ae9b153b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:19.626397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e126ca-c156-469a-977f-516ae9b153b4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:19.632204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:19.632204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:19.632429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:19.632736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] PATH_INFO: `/v3/projects/503ced8fc9834dc5b4e6dcca6a37267d` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:19.633970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=503ced8fc9834dc5b4e6dcca6a37267d)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:19.639648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:19.663074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:19.759201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '503ced8fc9834dc5b4e6dcca6a37267d', 'request_id': 'req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:19.765925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '503ced8fc9834dc5b4e6dcca6a37267d', 'request_id': 'req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:19.859083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '503ced8fc9834dc5b4e6dcca6a37267d', 'request_id': 'req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:19.873923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] The token cache is being invalidate because project 503ced8fc9834dc5b4e6dcca6a37267d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:19.874255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:19.876901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b12b180-d76f-4dd8-9e2c-53a9bbd12181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:19.879054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 989/1987] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:19 2022] DELETE /identity/v3/projects/503ced8fc9834dc5b4e6dcca6a37267d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:19.882839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97090b4e-cf34-4dde-87a9-d04e8e49695d service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:19.902826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97090b4e-cf34-4dde-87a9-d04e8e49695d service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:19.967948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:19.968735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:19.969014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:19.969284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] PATH_INFO: `/v3/users/5174cb621c4e4f8692ebd4349edca2c7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:19.970621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5174cb621c4e4f8692ebd4349edca2c7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:19.974326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:20.130630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5174cb621c4e4f8692ebd4349edca2c7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:20.214494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5174cb621c4e4f8692ebd4349edca2c7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:20.224960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5174cb621c4e4f8692ebd4349edca2c7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:20.225688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5174cb621c4e4f8692ebd4349edca2c7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:20.226391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5174cb621c4e4f8692ebd4349edca2c7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:20.253443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 999/1988] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:19 2022] DELETE /identity/v3/users/5174cb621c4e4f8692ebd4349edca2c7 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:21.157677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6a7f1-2f87-472c-94f9-20131e43f3e1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:21.158870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6a7f1-2f87-472c-94f9-20131e43f3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:21.159398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6a7f1-2f87-472c-94f9-20131e43f3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:21.216489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:21.217398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:21.232427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b6b6a7f1-2f87-472c-94f9-20131e43f3e1 None None] MFA Rules not processed for user `b7d7f6721a3a40f380578391949c443b`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:21.236448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b6b6a7f1-2f87-472c-94f9-20131e43f3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:21.340685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294addff-d7a8-4d51-a3cf-6c94fc5a15c6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:21.352441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:21.353138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:21.353415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:21.354403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] PATH_INFO: `/v3/projects/296615f4b50e4903bd90bfe2ce22be8d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:21.355586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=296615f4b50e4903bd90bfe2ce22be8d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:21.357670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:21.359686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 990/1989] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:21.359876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:21.364195 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6a7f1-2f87-472c-94f9-20131e43f3e1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:21.365877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b6b6a7f1-2f87-472c-94f9-20131e43f3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:21.397313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6a7f1-2f87-472c-94f9-20131e43f3e1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:21.402316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7732a9-7859-4a46-ab9c-776cf8d5f86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:21.403036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7732a9-7859-4a46-ab9c-776cf8d5f86f admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:21.403315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7732a9-7859-4a46-ab9c-776cf8d5f86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:21.403618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7732a9-7859-4a46-ab9c-776cf8d5f86f admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:21.404266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7732a9-7859-4a46-ab9c-776cf8d5f86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:21.406340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7732a9-7859-4a46-ab9c-776cf8d5f86f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:21.486143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296615f4b50e4903bd90bfe2ce22be8d', 'request_id': 'req-b686511a-05b8-4e0c-b3ec-15027ecbd124'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.495908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 991/1990] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:21 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:21.499258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296615f4b50e4903bd90bfe2ce22be8d', 'request_id': 'req-b686511a-05b8-4e0c-b3ec-15027ecbd124'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.500266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7732a9-7859-4a46-ab9c-776cf8d5f86f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:21.572903 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7732a9-7859-4a46-ab9c-776cf8d5f86f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:21.579388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296615f4b50e4903bd90bfe2ce22be8d', 'request_id': 'req-b686511a-05b8-4e0c-b3ec-15027ecbd124'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.581381 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86800e84-f16c-4327-a7eb-1bb3bbd84fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:21.582143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86800e84-f16c-4327-a7eb-1bb3bbd84fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:21.582417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86800e84-f16c-4327-a7eb-1bb3bbd84fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:21.582763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86800e84-f16c-4327-a7eb-1bb3bbd84fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:21.583298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86800e84-f16c-4327-a7eb-1bb3bbd84fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:21.585278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86800e84-f16c-4327-a7eb-1bb3bbd84fc1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:21.595674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] The token cache is being invalidate because project 296615f4b50e4903bd90bfe2ce22be8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:21.596356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.597435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.599109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1000/1991] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:21 2022] DELETE /identity/v3/projects/296615f4b50e4903bd90bfe2ce22be8d => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:21.605106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:21.606762 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:21.675883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686511a-05b8-4e0c-b3ec-15027ecbd124 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:21.681336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:21.681988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:21.682257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:21.682523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] PATH_INFO: `/v3/users/18197194fa60410f9050f877e314b0d5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:21.683734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=18197194fa60410f9050f877e314b0d5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:21.685751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:21.701437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 992/1992] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:21 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:21.704728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86800e84-f16c-4327-a7eb-1bb3bbd84fc1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:21.707477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-86800e84-f16c-4327-a7eb-1bb3bbd84fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:21.717238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:21.721323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86800e84-f16c-4327-a7eb-1bb3bbd84fc1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:21.726505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f2b32-a02b-44d2-ba48-a34d3a21d46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:21.730462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f2b32-a02b-44d2-ba48-a34d3a21d46c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:21.730462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f2b32-a02b-44d2-ba48-a34d3a21d46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:21.730462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f2b32-a02b-44d2-ba48-a34d3a21d46c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:21.730462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f2b32-a02b-44d2-ba48-a34d3a21d46c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:21.730897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f2b32-a02b-44d2-ba48-a34d3a21d46c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:21.741360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 993/1993] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:21.745343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f2b32-a02b-44d2-ba48-a34d3a21d46c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:21.756048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f2b32-a02b-44d2-ba48-a34d3a21d46c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:21.762063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24542dc-a80a-425d-bc6f-369caa849e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:21.762725 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24542dc-a80a-425d-bc6f-369caa849e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:21.763005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24542dc-a80a-425d-bc6f-369caa849e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:21.763276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24542dc-a80a-425d-bc6f-369caa849e2f admin admin] PATH_INFO: `/v3/projects/8d3fe8dee5294dddba8a43e1e3a4a45f/users/eee2e0d3e78f41ec83e2905623c46f1c/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:21.781331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24542dc-a80a-425d-bc6f-369caa849e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3fe8dee5294dddba8a43e1e3a4a45f, user_id=eee2e0d3e78f41ec83e2905623c46f1c, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:21.784075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24542dc-a80a-425d-bc6f-369caa849e2f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:21.883686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 994/1994] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:21 2022] PUT /identity/v3/projects/8d3fe8dee5294dddba8a43e1e3a4a45f/users/eee2e0d3e78f41ec83e2905623c46f1c/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:21.887675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24542dc-a80a-425d-bc6f-369caa849e2f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:21.893087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18197194fa60410f9050f877e314b0d5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.955928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18197194fa60410f9050f877e314b0d5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.957158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24542dc-a80a-425d-bc6f-369caa849e2f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:21.962203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc201c-bbae-4000-9724-de2f372199b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:21.962813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc201c-bbae-4000-9724-de2f372199b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:21.963684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc201c-bbae-4000-9724-de2f372199b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:21.967837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc201c-bbae-4000-9724-de2f372199b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:21.967837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc201c-bbae-4000-9724-de2f372199b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:21.968015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18197194fa60410f9050f877e314b0d5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.968110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc201c-bbae-4000-9724-de2f372199b3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:21.968192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18197194fa60410f9050f877e314b0d5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.968192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18197194fa60410f9050f877e314b0d5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:21.980975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 995/1995] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:21.982261 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc201c-bbae-4000-9724-de2f372199b3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:21.984127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-83bc201c-bbae-4000-9724-de2f372199b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:21.988642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1001/1996] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:21 2022] DELETE /identity/v3/users/18197194fa60410f9050f877e314b0d5 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:21.998314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc201c-bbae-4000-9724-de2f372199b3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:22.091091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeabb15-0c34-44a6-b8c4-9d07d297f153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:22.092944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeabb15-0c34-44a6-b8c4-9d07d297f153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:22.092944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeabb15-0c34-44a6-b8c4-9d07d297f153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:22.092944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeabb15-0c34-44a6-b8c4-9d07d297f153 admin admin] PATH_INFO: `/v3/projects/8d3fe8dee5294dddba8a43e1e3a4a45f/users/eee2e0d3e78f41ec83e2905623c46f1c/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:22.094986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeabb15-0c34-44a6-b8c4-9d07d297f153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3fe8dee5294dddba8a43e1e3a4a45f, user_id=eee2e0d3e78f41ec83e2905623c46f1c, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:22.097770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeabb15-0c34-44a6-b8c4-9d07d297f153 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:22.109553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 996/1997] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:21 2022] PUT /identity/v3/projects/8d3fe8dee5294dddba8a43e1e3a4a45f/users/eee2e0d3e78f41ec83e2905623c46f1c/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:22.624900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:22.626856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:22.653014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eeb91-84b6-49a6-a099-6c54c07e2a6c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:22.751704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:22.752320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:22.752669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:22.752941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] PATH_INFO: `/v3/projects/8a0a7f2c8a8e42efbfa8fd7f42e3157c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:22.754059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a0a7f2c8a8e42efbfa8fd7f42e3157c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:22.755979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:22.904744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a0a7f2c8a8e42efbfa8fd7f42e3157c', 'request_id': 'req-7b966760-3a22-431a-8fb9-47396bf8adb0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:22.912493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a0a7f2c8a8e42efbfa8fd7f42e3157c', 'request_id': 'req-7b966760-3a22-431a-8fb9-47396bf8adb0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:22.990887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a0a7f2c8a8e42efbfa8fd7f42e3157c', 'request_id': 'req-7b966760-3a22-431a-8fb9-47396bf8adb0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:23.005064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] The token cache is being invalidate because project 8a0a7f2c8a8e42efbfa8fd7f42e3157c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:23.005387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:23.006044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7b966760-3a22-431a-8fb9-47396bf8adb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:23.007582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1002/1998] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:22 2022] DELETE /identity/v3/projects/8a0a7f2c8a8e42efbfa8fd7f42e3157c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:23.014172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd7b0f-cf3b-4cca-a996-3c979e1c8989 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:23.014686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd7b0f-cf3b-4cca-a996-3c979e1c8989 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:23.014882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd7b0f-cf3b-4cca-a996-3c979e1c8989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:23.035747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-55fd7b0f-cf3b-4cca-a996-3c979e1c8989 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:23.037062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-55fd7b0f-cf3b-4cca-a996-3c979e1c8989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:23.113617 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 997/1999] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:23.123070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d2cff-2a76-4243-9f13-a9bc90ffc43f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:23.123452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d2cff-2a76-4243-9f13-a9bc90ffc43f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:23.123613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d2cff-2a76-4243-9f13-a9bc90ffc43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:23.152678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-467d2cff-2a76-4243-9f13-a9bc90ffc43f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:23.153832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-467d2cff-2a76-4243-9f13-a9bc90ffc43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:23.163257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1003/2000] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:23.167667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd7b0f-cf3b-4cca-a996-3c979e1c8989 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:23.182624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd7b0f-cf3b-4cca-a996-3c979e1c8989 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:23.187789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafb200-8429-4741-88c4-1f5c873d8c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:23.188448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafb200-8429-4741-88c4-1f5c873d8c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:23.188759 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafb200-8429-4741-88c4-1f5c873d8c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:23.189051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafb200-8429-4741-88c4-1f5c873d8c27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:23.189691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafb200-8429-4741-88c4-1f5c873d8c27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:23.189984 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafb200-8429-4741-88c4-1f5c873d8c27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:23.191589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafb200-8429-4741-88c4-1f5c873d8c27 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:23.204624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 998/2001] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:23.209216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d2cff-2a76-4243-9f13-a9bc90ffc43f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:23.220006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d2cff-2a76-4243-9f13-a9bc90ffc43f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:23.224942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013c447-389f-4baf-81c7-fa2195bade35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:23.225539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013c447-389f-4baf-81c7-fa2195bade35 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:23.225827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013c447-389f-4baf-81c7-fa2195bade35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:23.226098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013c447-389f-4baf-81c7-fa2195bade35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:23.226726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013c447-389f-4baf-81c7-fa2195bade35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:23.229307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013c447-389f-4baf-81c7-fa2195bade35 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:23.321260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1004/2002] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:23 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:23.328519 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafb200-8429-4741-88c4-1f5c873d8c27 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:23.340661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafb200-8429-4741-88c4-1f5c873d8c27 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:23.397166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:23.405358 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9aa0e-1753-4f54-83cc-354f72c645e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:23.406030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9aa0e-1753-4f54-83cc-354f72c645e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:23.406419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9aa0e-1753-4f54-83cc-354f72c645e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:23.406688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9aa0e-1753-4f54-83cc-354f72c645e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:23.407306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9aa0e-1753-4f54-83cc-354f72c645e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:23.409375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9aa0e-1753-4f54-83cc-354f72c645e5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:23.545494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 999/2003] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:23 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:23.549864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013c447-389f-4baf-81c7-fa2195bade35 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:23.561355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013c447-389f-4baf-81c7-fa2195bade35 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:23.566138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e102e10-4fe5-45ab-9814-1ae8e334893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:23.566764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e102e10-4fe5-45ab-9814-1ae8e334893d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:23.567035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e102e10-4fe5-45ab-9814-1ae8e334893d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:23.567297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e102e10-4fe5-45ab-9814-1ae8e334893d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:23.568028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e102e10-4fe5-45ab-9814-1ae8e334893d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:23.569671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e102e10-4fe5-45ab-9814-1ae8e334893d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:23.581550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1005/2004] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:23.585647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9aa0e-1753-4f54-83cc-354f72c645e5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:23.598545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9aa0e-1753-4f54-83cc-354f72c645e5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:23.603865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda924b-5d18-47d2-9986-d02170abf9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:23.604616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda924b-5d18-47d2-9986-d02170abf9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:23.605301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda924b-5d18-47d2-9986-d02170abf9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:23.605924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda924b-5d18-47d2-9986-d02170abf9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:23.607123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda924b-5d18-47d2-9986-d02170abf9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:23.609181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda924b-5d18-47d2-9986-d02170abf9a3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:23.636932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1000/2005] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:23.640117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e102e10-4fe5-45ab-9814-1ae8e334893d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:23.654086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e102e10-4fe5-45ab-9814-1ae8e334893d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:23.659126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca59bd0-b8ab-474d-8f57-c83bb4590e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:23.659780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca59bd0-b8ab-474d-8f57-c83bb4590e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:23.660704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca59bd0-b8ab-474d-8f57-c83bb4590e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:23.660704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca59bd0-b8ab-474d-8f57-c83bb4590e22 admin admin] PATH_INFO: `/v3/projects/935c99f110874341a8e939a33f06197e/users/e3ccf5bae2b24366a37da73017fac2f6/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:23.676254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca59bd0-b8ab-474d-8f57-c83bb4590e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=935c99f110874341a8e939a33f06197e, user_id=e3ccf5bae2b24366a37da73017fac2f6, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:23.681300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca59bd0-b8ab-474d-8f57-c83bb4590e22 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:23.771070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:23.778152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1006/2006] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:23 2022] PUT /identity/v3/projects/935c99f110874341a8e939a33f06197e/users/e3ccf5bae2b24366a37da73017fac2f6/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:23.792756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5ab9d-8d97-46db-a94f-91d6fe6a7fd8 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:23.792756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5ab9d-8d97-46db-a94f-91d6fe6a7fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:23.792756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5ab9d-8d97-46db-a94f-91d6fe6a7fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:23.843411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cef5ab9d-8d97-46db-a94f-91d6fe6a7fd8 None None] MFA Rules not processed for user `e3ccf5bae2b24366a37da73017fac2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:23.845344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cef5ab9d-8d97-46db-a94f-91d6fe6a7fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:23.921959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca59bd0-b8ab-474d-8f57-c83bb4590e22 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:23.927127 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4ca59bd0-b8ab-474d-8f57-c83bb4590e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:23.972014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1001/2007] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:23.978385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5ab9d-8d97-46db-a94f-91d6fe6a7fd8 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:24.033292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca59bd0-b8ab-474d-8f57-c83bb4590e22 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:24.038533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:24.039152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:24.039429 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:24.039737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] PATH_INFO: `/v3/users/7bc24f38528d4cf6b60734126def9447` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:24.041307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bc24f38528d4cf6b60734126def9447)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:24.047601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:24.070899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5ab9d-8d97-46db-a94f-91d6fe6a7fd8 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:24.084333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2d1ea-eb54-423d-977b-b79469f6b2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:24.085243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2d1ea-eb54-423d-977b-b79469f6b2dc admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:24.085675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2d1ea-eb54-423d-977b-b79469f6b2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:24.086097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2d1ea-eb54-423d-977b-b79469f6b2dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:24.086898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2d1ea-eb54-423d-977b-b79469f6b2dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:24.089059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2d1ea-eb54-423d-977b-b79469f6b2dc admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:24.249858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1002/2008] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:23 2022] POST /identity/v3/projects => generated 383 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:24.250904 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2d1ea-eb54-423d-977b-b79469f6b2dc admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:24.358099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bc24f38528d4cf6b60734126def9447'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:24.359043 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2d1ea-eb54-423d-977b-b79469f6b2dc admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:24.367071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdca4c-7168-4015-bf1f-2ebf45f6f43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:24.367695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdca4c-7168-4015-bf1f-2ebf45f6f43a admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:24.367970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdca4c-7168-4015-bf1f-2ebf45f6f43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:24.368240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdca4c-7168-4015-bf1f-2ebf45f6f43a admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:24.368947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfdca4c-7168-4015-bf1f-2ebf45f6f43a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:24.371171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfdca4c-7168-4015-bf1f-2ebf45f6f43a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:24.465031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bc24f38528d4cf6b60734126def9447'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:24.475224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bc24f38528d4cf6b60734126def9447'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:24.476895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bc24f38528d4cf6b60734126def9447'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:24.477656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a1f53b9b-383f-43e7-a7fd-8f24e661792c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bc24f38528d4cf6b60734126def9447'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:24.512876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1003/2009] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:24 2022] POST /identity/v3/users => generated 298 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:24.513736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdca4c-7168-4015-bf1f-2ebf45f6f43a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:24.518536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bdfdca4c-7168-4015-bf1f-2ebf45f6f43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:24.519961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1007/2010] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:23 2022] DELETE /identity/v3/users/7bc24f38528d4cf6b60734126def9447 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:24.524234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38ed0e-3e05-4dde-99d2-f1addc651ffb None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:24.524538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38ed0e-3e05-4dde-99d2-f1addc651ffb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:24.524811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38ed0e-3e05-4dde-99d2-f1addc651ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:24.545956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdca4c-7168-4015-bf1f-2ebf45f6f43a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:24.569882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3a38ed0e-3e05-4dde-99d2-f1addc651ffb None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:24.571138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3a38ed0e-3e05-4dde-99d2-f1addc651ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:24.678996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f53733-5efd-4acf-b1ae-a4e892150e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:24.682397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f53733-5efd-4acf-b1ae-a4e892150e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:24.682397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f53733-5efd-4acf-b1ae-a4e892150e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:24.682397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f53733-5efd-4acf-b1ae-a4e892150e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:24.682397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f53733-5efd-4acf-b1ae-a4e892150e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:24.683931 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f53733-5efd-4acf-b1ae-a4e892150e97 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:24.712137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1004/2011] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:24.717094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1008/2012] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:24.718277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f53733-5efd-4acf-b1ae-a4e892150e97 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:24.725606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38ed0e-3e05-4dde-99d2-f1addc651ffb None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:24.727297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3a38ed0e-3e05-4dde-99d2-f1addc651ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:24.733967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f53733-5efd-4acf-b1ae-a4e892150e97 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:24.744936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602a05c-ea46-46aa-aabd-06b55a2049ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:24.744936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602a05c-ea46-46aa-aabd-06b55a2049ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:24.744936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602a05c-ea46-46aa-aabd-06b55a2049ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:24.744936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602a05c-ea46-46aa-aabd-06b55a2049ef admin admin] PATH_INFO: `/v3/projects/b38ac668cddf4f51b2f2cad55bdbe821/users/b5b8835de81047d7b5157197df9db1b6/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:24.772916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602a05c-ea46-46aa-aabd-06b55a2049ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38ac668cddf4f51b2f2cad55bdbe821, user_id=b5b8835de81047d7b5157197df9db1b6, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:24.779171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602a05c-ea46-46aa-aabd-06b55a2049ef admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:24.825441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38ed0e-3e05-4dde-99d2-f1addc651ffb None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:24.829362 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25771f7-e5de-4e03-b43f-46c4e39ca1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:24.829990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25771f7-e5de-4e03-b43f-46c4e39ca1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:24.830567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25771f7-e5de-4e03-b43f-46c4e39ca1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:24.830567 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25771f7-e5de-4e03-b43f-46c4e39ca1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:24.842393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25771f7-e5de-4e03-b43f-46c4e39ca1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:24.843888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25771f7-e5de-4e03-b43f-46c4e39ca1ee service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:24.856196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1009/2013] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:24.881010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1005/2014] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:24 2022] PUT /identity/v3/projects/b38ac668cddf4f51b2f2cad55bdbe821/users/b5b8835de81047d7b5157197df9db1b6/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:24.885256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25771f7-e5de-4e03-b43f-46c4e39ca1ee service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:24.899634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25771f7-e5de-4e03-b43f-46c4e39ca1ee service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:25.013158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48f290-7b46-488c-b6b7-3d0a5e39a71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:25.013574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48f290-7b46-488c-b6b7-3d0a5e39a71a admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:25.014024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48f290-7b46-488c-b6b7-3d0a5e39a71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:25.014464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48f290-7b46-488c-b6b7-3d0a5e39a71a admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:25.015273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48f290-7b46-488c-b6b7-3d0a5e39a71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:25.017303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48f290-7b46-488c-b6b7-3d0a5e39a71a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:25.033798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1010/2015] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:25.038128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602a05c-ea46-46aa-aabd-06b55a2049ef admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:25.051927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602a05c-ea46-46aa-aabd-06b55a2049ef admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:25.054889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:25.061013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a50e4-3dfc-4794-92f5-1d4d383829e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:25.061953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a50e4-3dfc-4794-92f5-1d4d383829e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:25.062419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a50e4-3dfc-4794-92f5-1d4d383829e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:25.062857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a50e4-3dfc-4794-92f5-1d4d383829e9 admin admin] PATH_INFO: `/v3/projects/b38ac668cddf4f51b2f2cad55bdbe821/users/b5b8835de81047d7b5157197df9db1b6/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:25.065590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a50e4-3dfc-4794-92f5-1d4d383829e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38ac668cddf4f51b2f2cad55bdbe821, user_id=b5b8835de81047d7b5157197df9db1b6, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:25.068633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a50e4-3dfc-4794-92f5-1d4d383829e9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:25.086744 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1006/2016] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:25 2022] PUT /identity/v3/projects/b38ac668cddf4f51b2f2cad55bdbe821/users/b5b8835de81047d7b5157197df9db1b6/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:25.098186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c1115-8ce6-4c01-bfc4-1134b5516879 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:25.098476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c1115-8ce6-4c01-bfc4-1134b5516879 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:25.099855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c1115-8ce6-4c01-bfc4-1134b5516879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:25.152866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0a7c1115-8ce6-4c01-bfc4-1134b5516879 None None] MFA Rules not processed for user `b5b8835de81047d7b5157197df9db1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:25.157113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0a7c1115-8ce6-4c01-bfc4-1134b5516879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:25.210192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:25.304670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1011/2017] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:25.314446 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a50e4-3dfc-4794-92f5-1d4d383829e9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:25.317314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4b7a50e4-3dfc-4794-92f5-1d4d383829e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:25.419413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a50e4-3dfc-4794-92f5-1d4d383829e9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:25.423636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54e0f9-a5a2-4033-81fb-5bca7ebbcb3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:25.428361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54e0f9-a5a2-4033-81fb-5bca7ebbcb3f service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:25.428361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54e0f9-a5a2-4033-81fb-5bca7ebbcb3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:25.428361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54e0f9-a5a2-4033-81fb-5bca7ebbcb3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:25.444756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54e0f9-a5a2-4033-81fb-5bca7ebbcb3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:25.446773 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54e0f9-a5a2-4033-81fb-5bca7ebbcb3f service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:25.461700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1007/2018] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:25.567425 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c1115-8ce6-4c01-bfc4-1134b5516879 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:25.568974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0a7c1115-8ce6-4c01-bfc4-1134b5516879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:25.736655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c1115-8ce6-4c01-bfc4-1134b5516879 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:25.749066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54e0f9-a5a2-4033-81fb-5bca7ebbcb3f service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:25.751150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8d54e0f9-a5a2-4033-81fb-5bca7ebbcb3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:25.754906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:25.755885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:25.756410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:25.757074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] PATH_INFO: `/v3/projects/aaf010b6bea84e77a58a124a7592ccca` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:25.758538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf010b6bea84e77a58a124a7592ccca)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:25.766004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:25.768128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54e0f9-a5a2-4033-81fb-5bca7ebbcb3f service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:25.785230 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e5ae2-0991-4fca-a8b8-cf57acfb592e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:25.786006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e5ae2-0991-4fca-a8b8-cf57acfb592e service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:25.786404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e5ae2-0991-4fca-a8b8-cf57acfb592e service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:25.786730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e5ae2-0991-4fca-a8b8-cf57acfb592e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:25.791865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a39e5ae2-0991-4fca-a8b8-cf57acfb592e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:25.869249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e5ae2-0991-4fca-a8b8-cf57acfb592e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:25.871188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e5ae2-0991-4fca-a8b8-cf57acfb592e service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:25.880873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1008/2019] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:25.888275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf010b6bea84e77a58a124a7592ccca', 'request_id': 'req-b5a21e13-199e-483c-a7b2-84c105454c9d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:25.897979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf010b6bea84e77a58a124a7592ccca', 'request_id': 'req-b5a21e13-199e-483c-a7b2-84c105454c9d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:25.967217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf010b6bea84e77a58a124a7592ccca', 'request_id': 'req-b5a21e13-199e-483c-a7b2-84c105454c9d'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:25.980406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] The token cache is being invalidate because project aaf010b6bea84e77a58a124a7592ccca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:25.980978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:25.981730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:25.983751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1012/2020] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:25 2022] DELETE /identity/v3/projects/aaf010b6bea84e77a58a124a7592ccca => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:25.987711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e5ae2-0991-4fca-a8b8-cf57acfb592e service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:25.999400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e5ae2-0991-4fca-a8b8-cf57acfb592e service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:26.062677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:26.063354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:26.063624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:26.063900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] PATH_INFO: `/v3/users/d3ab9d719f05495b8ce0b0dc080838b4` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:26.065058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ab9d719f05495b8ce0b0dc080838b4)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:26.066988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:26.236081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:26.365872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ab9d719f05495b8ce0b0dc080838b4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.524989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ab9d719f05495b8ce0b0dc080838b4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.566983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ab9d719f05495b8ce0b0dc080838b4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.568094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ab9d719f05495b8ce0b0dc080838b4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.569072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3c4c65e5-72b0-46d0-a2df-c156b8009da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ab9d719f05495b8ce0b0dc080838b4'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.570307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:26.572004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:26.588085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a21e13-199e-483c-a7b2-84c105454c9d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:26.594083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1009/2021] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:25 2022] DELETE /identity/v3/users/d3ab9d719f05495b8ce0b0dc080838b4 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:26.594221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:26.596405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:26.596405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:26.596405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] PATH_INFO: `/v3/users/e3ccf5bae2b24366a37da73017fac2f6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:26.596405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ccf5bae2b24366a37da73017fac2f6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:26.599966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:26.813449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ccf5bae2b24366a37da73017fac2f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.897979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ccf5bae2b24366a37da73017fac2f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.906473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ccf5bae2b24366a37da73017fac2f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.907245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ccf5bae2b24366a37da73017fac2f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.907861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ccf5bae2b24366a37da73017fac2f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:26.946042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1013/2022] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:26 2022] DELETE /identity/v3/users/e3ccf5bae2b24366a37da73017fac2f6 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:26.954874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7197e8d-307b-45e8-b1a7-0dbe1e1368f8 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:26.955420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7197e8d-307b-45e8-b1a7-0dbe1e1368f8 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:26.955884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7197e8d-307b-45e8-b1a7-0dbe1e1368f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:26.979069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b7197e8d-307b-45e8-b1a7-0dbe1e1368f8 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:26.980288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b7197e8d-307b-45e8-b1a7-0dbe1e1368f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:27.050385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:27.130057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1010/2023] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:27.160732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:27.162208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:27.235916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee548e-b60a-4731-a77e-e90fa027327e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:27.239716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb5268-9526-46bb-a909-54567eae286c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:27.240346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb5268-9526-46bb-a909-54567eae286c service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:27.240660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb5268-9526-46bb-a909-54567eae286c service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:27.240927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb5268-9526-46bb-a909-54567eae286c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:27.255323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7197e8d-307b-45e8-b1a7-0dbe1e1368f8 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:27.258324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b7197e8d-307b-45e8-b1a7-0dbe1e1368f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:27.271546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb5268-9526-46bb-a909-54567eae286c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:27.273572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb5268-9526-46bb-a909-54567eae286c service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:27.283696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7197e8d-307b-45e8-b1a7-0dbe1e1368f8 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:27.297179 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1014/2024] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:27.314074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:27.315025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:27.315523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:27.323638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] PATH_INFO: `/v3/projects/93637ab846764fe196da20c2c274d93f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:27.323638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93637ab846764fe196da20c2c274d93f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:27.323638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:27.414266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:27.546546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93637ab846764fe196da20c2c274d93f', 'request_id': 'req-45ccda6e-6819-429a-b4d5-441c701c9779'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:27.586543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93637ab846764fe196da20c2c274d93f', 'request_id': 'req-45ccda6e-6819-429a-b4d5-441c701c9779'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:27.653230 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93637ab846764fe196da20c2c274d93f', 'request_id': 'req-45ccda6e-6819-429a-b4d5-441c701c9779'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:27.667908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] The token cache is being invalidate because project 93637ab846764fe196da20c2c274d93f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:27.668455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:27.669349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-45ccda6e-6819-429a-b4d5-441c701c9779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:27.671712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1011/2025] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:27 2022] DELETE /identity/v3/projects/93637ab846764fe196da20c2c274d93f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:27.676833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008d1e5-15ba-42c8-a81b-6a2e2b8293e9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:27.677361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008d1e5-15ba-42c8-a81b-6a2e2b8293e9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:27.678040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008d1e5-15ba-42c8-a81b-6a2e2b8293e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:27.715527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c008d1e5-15ba-42c8-a81b-6a2e2b8293e9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:27.718026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c008d1e5-15ba-42c8-a81b-6a2e2b8293e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:27.812280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1015/2026] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:27.819841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99225eaf-6b53-4335-86eb-aa58392902c6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:27.820422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99225eaf-6b53-4335-86eb-aa58392902c6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:27.821047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99225eaf-6b53-4335-86eb-aa58392902c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:27.848399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-99225eaf-6b53-4335-86eb-aa58392902c6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:27.850711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-99225eaf-6b53-4335-86eb-aa58392902c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:27.862757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1012/2027] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:27.867309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008d1e5-15ba-42c8-a81b-6a2e2b8293e9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:27.898104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008d1e5-15ba-42c8-a81b-6a2e2b8293e9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:27.901820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:27.901820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:27.902171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:27.902388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:27.903006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:27.903304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:27.906910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:27.927293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1016/2028] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:10:27.928283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99225eaf-6b53-4335-86eb-aa58392902c6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:27.938421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99225eaf-6b53-4335-86eb-aa58392902c6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:27.943978 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55692a73-f02a-4acb-af26-6e353cd8940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:27.944623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55692a73-f02a-4acb-af26-6e353cd8940d admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:27.944902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55692a73-f02a-4acb-af26-6e353cd8940d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:27.945915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55692a73-f02a-4acb-af26-6e353cd8940d admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:27.946559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55692a73-f02a-4acb-af26-6e353cd8940d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:27.949449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55692a73-f02a-4acb-af26-6e353cd8940d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:27.974759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:27.997882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:28.008723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b66247-14d9-4c50-9023-bd3394c2025f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:28.016664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:28.019741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:28.020028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:28.020298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] PATH_INFO: `/v3/projects/935c99f110874341a8e939a33f06197e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:28.022299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935c99f110874341a8e939a33f06197e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:28.026465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:28.051820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1013/2029] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:27 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:28.055997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55692a73-f02a-4acb-af26-6e353cd8940d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:28.126114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55692a73-f02a-4acb-af26-6e353cd8940d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:28.131997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d97d3c-ee0e-4aab-b638-5ed8da5ea97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:28.132695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d97d3c-ee0e-4aab-b638-5ed8da5ea97c admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:28.132976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d97d3c-ee0e-4aab-b638-5ed8da5ea97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:28.133248 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d97d3c-ee0e-4aab-b638-5ed8da5ea97c admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:28.133884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d97d3c-ee0e-4aab-b638-5ed8da5ea97c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:28.140926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d97d3c-ee0e-4aab-b638-5ed8da5ea97c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:28.184229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935c99f110874341a8e939a33f06197e', 'request_id': 'req-daa008af-846d-4150-b34c-70e7727c1507'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.199884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935c99f110874341a8e939a33f06197e', 'request_id': 'req-daa008af-846d-4150-b34c-70e7727c1507'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.300473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1014/2030] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:28 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:28.304992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d97d3c-ee0e-4aab-b638-5ed8da5ea97c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:28.315174 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d97d3c-ee0e-4aab-b638-5ed8da5ea97c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:28.323432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935c99f110874341a8e939a33f06197e', 'request_id': 'req-daa008af-846d-4150-b34c-70e7727c1507'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.344839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] The token cache is being invalidate because project 935c99f110874341a8e939a33f06197e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:28.345444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.346539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.351791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1017/2031] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:27 2022] DELETE /identity/v3/projects/935c99f110874341a8e939a33f06197e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:28.352968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:28.355093 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:28.406976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c9b10-fd24-4d77-a101-d022a137b4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:28.406976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c9b10-fd24-4d77-a101-d022a137b4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:28.407245 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c9b10-fd24-4d77-a101-d022a137b4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:28.407443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c9b10-fd24-4d77-a101-d022a137b4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:28.408111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c9b10-fd24-4d77-a101-d022a137b4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:28.411422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c9b10-fd24-4d77-a101-d022a137b4f7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:28.432664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1015/2032] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:28.435322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c9b10-fd24-4d77-a101-d022a137b4f7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:28.437442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e04c9b10-fd24-4d77-a101-d022a137b4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:28.456162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c9b10-fd24-4d77-a101-d022a137b4f7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:28.473688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ade2bc-cdd6-44ed-b41b-0bc9c83002f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:28.475614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ade2bc-cdd6-44ed-b41b-0bc9c83002f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:28.476106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ade2bc-cdd6-44ed-b41b-0bc9c83002f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:28.477218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ade2bc-cdd6-44ed-b41b-0bc9c83002f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:28.478087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ade2bc-cdd6-44ed-b41b-0bc9c83002f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:28.481152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ade2bc-cdd6-44ed-b41b-0bc9c83002f5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:28.515343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1016/2033] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:28.517338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ade2bc-cdd6-44ed-b41b-0bc9c83002f5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:28.526099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa008af-846d-4150-b34c-70e7727c1507 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:28.531982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:28.532660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:28.532937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:28.533207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] PATH_INFO: `/v3/users/b5b8835de81047d7b5157197df9db1b6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:28.534403 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5b8835de81047d7b5157197df9db1b6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:28.536478 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ade2bc-cdd6-44ed-b41b-0bc9c83002f5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:28.537327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:28.545570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21a551-ac88-483e-a8fe-79557ae4066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:28.546388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21a551-ac88-483e-a8fe-79557ae4066c admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:28.546741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21a551-ac88-483e-a8fe-79557ae4066c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:28.547084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21a551-ac88-483e-a8fe-79557ae4066c admin admin] PATH_INFO: `/v3/projects/858c5b2903e64efaa1d5c7ca0d29d471/users/aa81de8e90fb4b1d867062853b058aaf/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:28.566490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21a551-ac88-483e-a8fe-79557ae4066c admin admin] RBAC: Authorizing `identity:create_grant(project_id=858c5b2903e64efaa1d5c7ca0d29d471, user_id=aa81de8e90fb4b1d867062853b058aaf, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:28.573616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21a551-ac88-483e-a8fe-79557ae4066c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:28.674462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1017/2034] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:28 2022] PUT /identity/v3/projects/858c5b2903e64efaa1d5c7ca0d29d471/users/aa81de8e90fb4b1d867062853b058aaf/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:28.685091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca451dac-b099-4d68-aca0-3cb33520c5ce None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:28.685557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca451dac-b099-4d68-aca0-3cb33520c5ce None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:28.685673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca451dac-b099-4d68-aca0-3cb33520c5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:28.718232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ca451dac-b099-4d68-aca0-3cb33520c5ce None None] MFA Rules not processed for user `aa81de8e90fb4b1d867062853b058aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:28.718792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ca451dac-b099-4d68-aca0-3cb33520c5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:28.744359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5b8835de81047d7b5157197df9db1b6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.817513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1018/2035] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:28.822670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca451dac-b099-4d68-aca0-3cb33520c5ce None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:28.832650 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:28.872295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5b8835de81047d7b5157197df9db1b6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.872295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5b8835de81047d7b5157197df9db1b6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.872295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5b8835de81047d7b5157197df9db1b6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.872295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5b8835de81047d7b5157197df9db1b6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:28.916778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1018/2036] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:28 2022] DELETE /identity/v3/users/b5b8835de81047d7b5157197df9db1b6 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:28.933648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca451dac-b099-4d68-aca0-3cb33520c5ce None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:29.012801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac572b9-f585-454c-86e6-14de9b2a0bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:29.013502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac572b9-f585-454c-86e6-14de9b2a0bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:29.018653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac572b9-f585-454c-86e6-14de9b2a0bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:29.018653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac572b9-f585-454c-86e6-14de9b2a0bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:29.018653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac572b9-f585-454c-86e6-14de9b2a0bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:29.018653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac572b9-f585-454c-86e6-14de9b2a0bc4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:29.065596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:29.137158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:29.137521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1019/2037] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:28 2022] POST /identity/v3/projects => generated 385 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:29.139155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:29.155121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e896c0-d5df-4ece-b749-9c40b04bbd67 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:29.220176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a2419-7a1c-4dba-822a-e5cd1d1961a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:29.220831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a2419-7a1c-4dba-822a-e5cd1d1961a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:29.221111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a2419-7a1c-4dba-822a-e5cd1d1961a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:29.221373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a2419-7a1c-4dba-822a-e5cd1d1961a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:29.222007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a2419-7a1c-4dba-822a-e5cd1d1961a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:29.223997 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a2419-7a1c-4dba-822a-e5cd1d1961a6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:29.344558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1019/2038] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:29 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:29.345951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac572b9-f585-454c-86e6-14de9b2a0bc4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:29.358407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac572b9-f585-454c-86e6-14de9b2a0bc4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:29.366157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46199de3-ef06-4258-b1d3-194dbbaf0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:29.366860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46199de3-ef06-4258-b1d3-194dbbaf0082 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:29.367311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46199de3-ef06-4258-b1d3-194dbbaf0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:29.384733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46199de3-ef06-4258-b1d3-194dbbaf0082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:29.384733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46199de3-ef06-4258-b1d3-194dbbaf0082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:29.384733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46199de3-ef06-4258-b1d3-194dbbaf0082 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:29.386279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1020/2039] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:29.390500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a2419-7a1c-4dba-822a-e5cd1d1961a6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:29.413427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a2419-7a1c-4dba-822a-e5cd1d1961a6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:29.423909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef1ef7-7971-4c63-bfb7-50e1241b4c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:29.424647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef1ef7-7971-4c63-bfb7-50e1241b4c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:29.425084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef1ef7-7971-4c63-bfb7-50e1241b4c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:29.425358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef1ef7-7971-4c63-bfb7-50e1241b4c10 admin admin] PATH_INFO: `/v3/projects/5fada3fb4a8b4dc6a223b78de4528e9b/users/fb2782f9e16a458b88e7dcdaec85b62e/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:29.447566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef1ef7-7971-4c63-bfb7-50e1241b4c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fada3fb4a8b4dc6a223b78de4528e9b, user_id=fb2782f9e16a458b88e7dcdaec85b62e, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:29.454874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef1ef7-7971-4c63-bfb7-50e1241b4c10 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:29.538098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1020/2040] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:29 2022] PUT /identity/v3/projects/5fada3fb4a8b4dc6a223b78de4528e9b/users/fb2782f9e16a458b88e7dcdaec85b62e/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:29.542453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46199de3-ef06-4258-b1d3-194dbbaf0082 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:29.552407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:29.555467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46199de3-ef06-4258-b1d3-194dbbaf0082 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:29.670197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd016-0db3-4572-8b3b-7943f780a712 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:29.670473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd016-0db3-4572-8b3b-7943f780a712 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:29.670737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd016-0db3-4572-8b3b-7943f780a712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:29.683643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07062b-f3be-4beb-a4c2-d97aeabb4119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:29.684266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07062b-f3be-4beb-a4c2-d97aeabb4119 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:29.684586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07062b-f3be-4beb-a4c2-d97aeabb4119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:29.685309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07062b-f3be-4beb-a4c2-d97aeabb4119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:29.685964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07062b-f3be-4beb-a4c2-d97aeabb4119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:29.687736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07062b-f3be-4beb-a4c2-d97aeabb4119 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:29.702382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1021/2041] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:29.707289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07062b-f3be-4beb-a4c2-d97aeabb4119 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:29.711255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0a8bd016-0db3-4572-8b3b-7943f780a712 None None] MFA Rules not processed for user `eee2e0d3e78f41ec83e2905623c46f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:29.712565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0a8bd016-0db3-4572-8b3b-7943f780a712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:29.720145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07062b-f3be-4beb-a4c2-d97aeabb4119 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:29.726682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54af9c0-59f0-4403-846d-da1c792937b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:29.727215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54af9c0-59f0-4403-846d-da1c792937b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:29.727464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54af9c0-59f0-4403-846d-da1c792937b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:29.727738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54af9c0-59f0-4403-846d-da1c792937b2 admin admin] PATH_INFO: `/v3/projects/5fada3fb4a8b4dc6a223b78de4528e9b/users/fb2782f9e16a458b88e7dcdaec85b62e/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:29.730372 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54af9c0-59f0-4403-846d-da1c792937b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fada3fb4a8b4dc6a223b78de4528e9b, user_id=fb2782f9e16a458b88e7dcdaec85b62e, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:29.733167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54af9c0-59f0-4403-846d-da1c792937b2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:29.751101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1022/2042] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:29 2022] PUT /identity/v3/projects/5fada3fb4a8b4dc6a223b78de4528e9b/users/fb2782f9e16a458b88e7dcdaec85b62e/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:29.752400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54af9c0-59f0-4403-846d-da1c792937b2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:29.760919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e54af9c0-59f0-4403-846d-da1c792937b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:29.823623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1021/2043] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:29.829882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb53fd-0738-4a74-8ba4-50ec07a3c2a6 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:29.830771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb53fd-0738-4a74-8ba4-50ec07a3c2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:29.831230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb53fd-0738-4a74-8ba4-50ec07a3c2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:29.857209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54af9c0-59f0-4403-846d-da1c792937b2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:29.862133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:29.862821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:29.863088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:29.863412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] PATH_INFO: `/v3/projects/b38ac668cddf4f51b2f2cad55bdbe821` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:29.864483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38ac668cddf4f51b2f2cad55bdbe821)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:29.866459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:29.879322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-faeb53fd-0738-4a74-8ba4-50ec07a3c2a6 None None] MFA Rules not processed for user `fb2782f9e16a458b88e7dcdaec85b62e`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:29.881184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-faeb53fd-0738-4a74-8ba4-50ec07a3c2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:29.963494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1022/2044] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:29.964648 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb53fd-0738-4a74-8ba4-50ec07a3c2a6 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:29.966414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-faeb53fd-0738-4a74-8ba4-50ec07a3c2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:29.999777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38ac668cddf4f51b2f2cad55bdbe821', 'request_id': 'req-ca05faf8-3d99-4d49-802c-f0bb6468ae11'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.005364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38ac668cddf4f51b2f2cad55bdbe821', 'request_id': 'req-ca05faf8-3d99-4d49-802c-f0bb6468ae11'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.073133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb53fd-0738-4a74-8ba4-50ec07a3c2a6 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.103922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38ac668cddf4f51b2f2cad55bdbe821', 'request_id': 'req-ca05faf8-3d99-4d49-802c-f0bb6468ae11'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.116398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] The token cache is being invalidate because project b38ac668cddf4f51b2f2cad55bdbe821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:30.116746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.117444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.119933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1023/2045] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:29 2022] DELETE /identity/v3/projects/b38ac668cddf4f51b2f2cad55bdbe821 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:30.120821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:30.122123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:30.199420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1c7a-4b9f-402a-bef5-a3972286d647 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.200047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1c7a-4b9f-402a-bef5-a3972286d647 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.200320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1c7a-4b9f-402a-bef5-a3972286d647 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.200615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1c7a-4b9f-402a-bef5-a3972286d647 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.202536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c4ae1c7a-4b9f-402a-bef5-a3972286d647 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:30.203362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05faf8-3d99-4d49-802c-f0bb6468ae11 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.207623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.208235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.208800 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.209073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.213266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:30.284657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae1c7a-4b9f-402a-bef5-a3972286d647 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:30.286336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae1c7a-4b9f-402a-bef5-a3972286d647 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:30.304724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1023/2046] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:30.308376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6406312-796c-4058-b1d5-ecbaaacc3400 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.310679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:30.311049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6406312-796c-4058-b1d5-ecbaaacc3400 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.311354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6406312-796c-4058-b1d5-ecbaaacc3400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.313809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:30.332641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1024/2047] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:30.352292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f6406312-796c-4058-b1d5-ecbaaacc3400 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:30.353599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f6406312-796c-4058-b1d5-ecbaaacc3400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:30.414819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:30.416528 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:30.427208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1024/2048] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:30.436296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf70dc-9747-442a-961c-ecb241d12226 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.436609 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf70dc-9747-442a-961c-ecb241d12226 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.436885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf70dc-9747-442a-961c-ecb241d12226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.463498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-02cf70dc-9747-442a-961c-ecb241d12226 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:30.464889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-02cf70dc-9747-442a-961c-ecb241d12226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:30.474269 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1025/2049] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:30.478889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf70dc-9747-442a-961c-ecb241d12226 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:30.495641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf70dc-9747-442a-961c-ecb241d12226 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.497371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84318eb-af77-4d49-b173-70a43e1667bd service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.500801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ade984-54b4-4e6b-8d77-0cecba9b36a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.501406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ade984-54b4-4e6b-8d77-0cecba9b36a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.501680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ade984-54b4-4e6b-8d77-0cecba9b36a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.501949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ade984-54b4-4e6b-8d77-0cecba9b36a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.502243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.502864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.502983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ade984-54b4-4e6b-8d77-0cecba9b36a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:30.503124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.503227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ade984-54b4-4e6b-8d77-0cecba9b36a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:30.503389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] PATH_INFO: `/v3/users/aa81de8e90fb4b1d867062853b058aaf` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.504897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa81de8e90fb4b1d867062853b058aaf)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:30.505026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ade984-54b4-4e6b-8d77-0cecba9b36a6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:30.506783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:30.518563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1026/2050] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:10:30.523109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ade984-54b4-4e6b-8d77-0cecba9b36a6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:30.533839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ade984-54b4-4e6b-8d77-0cecba9b36a6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.538717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae96ff3-140c-4024-ae77-66c5242add95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.539301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae96ff3-140c-4024-ae77-66c5242add95 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.539574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae96ff3-140c-4024-ae77-66c5242add95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.539838 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae96ff3-140c-4024-ae77-66c5242add95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.540442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae96ff3-140c-4024-ae77-66c5242add95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:30.542933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae96ff3-140c-4024-ae77-66c5242add95 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:30.660216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1027/2051] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:30 2022] POST /identity/v3/projects => generated 359 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:30.664268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae96ff3-140c-4024-ae77-66c5242add95 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:30.698477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa81de8e90fb4b1d867062853b058aaf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.727762 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae96ff3-140c-4024-ae77-66c5242add95 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.734899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06877715-5de8-468a-88f4-7aba01569dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.735501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877715-5de8-468a-88f4-7aba01569dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.735781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877715-5de8-468a-88f4-7aba01569dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.736048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877715-5de8-468a-88f4-7aba01569dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.736699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06877715-5de8-468a-88f4-7aba01569dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:30.738817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06877715-5de8-468a-88f4-7aba01569dd5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:30.772971 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa81de8e90fb4b1d867062853b058aaf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.781911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa81de8e90fb4b1d867062853b058aaf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.783578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa81de8e90fb4b1d867062853b058aaf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.783578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a590b75d-f5e0-4c5f-884a-e7b4b7501d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa81de8e90fb4b1d867062853b058aaf'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:30.803093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1025/2052] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:30 2022] DELETE /identity/v3/users/aa81de8e90fb4b1d867062853b058aaf => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:30.807657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9ecc4-f5a9-4d43-bfa2-927b787414f5 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.807941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9ecc4-f5a9-4d43-bfa2-927b787414f5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.808209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9ecc4-f5a9-4d43-bfa2-927b787414f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.828059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6ac9ecc4-f5a9-4d43-bfa2-927b787414f5 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:30.829298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6ac9ecc4-f5a9-4d43-bfa2-927b787414f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:30.840056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1028/2053] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:30 2022] POST /identity/v3/users => generated 280 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:30.841153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06877715-5de8-468a-88f4-7aba01569dd5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:30.842840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-06877715-5de8-468a-88f4-7aba01569dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:30.855217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06877715-5de8-468a-88f4-7aba01569dd5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.896674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1026/2054] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:30.907306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9ecc4-f5a9-4d43-bfa2-927b787414f5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:30.908684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6ac9ecc4-f5a9-4d43-bfa2-927b787414f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:30.916671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5edb2-947d-43df-b9e3-a0d563d12ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.917295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5edb2-947d-43df-b9e3-a0d563d12ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.917575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5edb2-947d-43df-b9e3-a0d563d12ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.917847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5edb2-947d-43df-b9e3-a0d563d12ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.918492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5edb2-947d-43df-b9e3-a0d563d12ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:30.920113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5edb2-947d-43df-b9e3-a0d563d12ab4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:30.930520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1029/2055] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:30.935694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5edb2-947d-43df-b9e3-a0d563d12ab4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:30.945553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5edb2-947d-43df-b9e3-a0d563d12ab4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.950787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23804c2-641c-4fb0-8e03-948d01c327f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.951376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23804c2-641c-4fb0-8e03-948d01c327f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.951689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23804c2-641c-4fb0-8e03-948d01c327f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.951951 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23804c2-641c-4fb0-8e03-948d01c327f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.952620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23804c2-641c-4fb0-8e03-948d01c327f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:30.954266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23804c2-641c-4fb0-8e03-948d01c327f6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:30.965783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9ecc4-f5a9-4d43-bfa2-927b787414f5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.968544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1030/2056] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:30.972899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23804c2-641c-4fb0-8e03-948d01c327f6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:30.973058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bb75c-247f-4889-b5fd-2cb22acba6db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.973551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bb75c-247f-4889-b5fd-2cb22acba6db service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.973826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bb75c-247f-4889-b5fd-2cb22acba6db service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.974129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bb75c-247f-4889-b5fd-2cb22acba6db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.984814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23804c2-641c-4fb0-8e03-948d01c327f6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:30.987693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bb75c-247f-4889-b5fd-2cb22acba6db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:30.989234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bb75c-247f-4889-b5fd-2cb22acba6db service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:30.995898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a690285-fcd9-472e-b055-a6c14edbac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:30.996586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a690285-fcd9-472e-b055-a6c14edbac74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:30.996989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a690285-fcd9-472e-b055-a6c14edbac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:30.997264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a690285-fcd9-472e-b055-a6c14edbac74 admin admin] PATH_INFO: `/v3/projects/cd3fadb8badc45f2a025ac771e294730/users/dcfe5d7a8aa549a3849f387fb5e3cdab/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:30.999261 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1027/2057] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:31.015056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a690285-fcd9-472e-b055-a6c14edbac74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3fadb8badc45f2a025ac771e294730, user_id=dcfe5d7a8aa549a3849f387fb5e3cdab, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:31.018927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a690285-fcd9-472e-b055-a6c14edbac74 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:31.099852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1031/2058] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:30 2022] PUT /identity/v3/projects/cd3fadb8badc45f2a025ac771e294730/users/dcfe5d7a8aa549a3849f387fb5e3cdab/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:31.110470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6616500-91f8-4a86-ae94-88a81f4087cd None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:31.110756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6616500-91f8-4a86-ae94-88a81f4087cd None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:31.111029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6616500-91f8-4a86-ae94-88a81f4087cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:31.141462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b6616500-91f8-4a86-ae94-88a81f4087cd None None] MFA Rules not processed for user `dcfe5d7a8aa549a3849f387fb5e3cdab`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:31.142734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b6616500-91f8-4a86-ae94-88a81f4087cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:31.233473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:31.241503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1028/2059] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:31.278115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a690285-fcd9-472e-b055-a6c14edbac74 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:31.279886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2a690285-fcd9-472e-b055-a6c14edbac74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:31.344383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a690285-fcd9-472e-b055-a6c14edbac74 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:31.348306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3bf73-6a3a-41b5-a830-149a6fd51f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:31.348952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3bf73-6a3a-41b5-a830-149a6fd51f19 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:31.349252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3bf73-6a3a-41b5-a830-149a6fd51f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:31.349575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3bf73-6a3a-41b5-a830-149a6fd51f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:31.365443 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3bf73-6a3a-41b5-a830-149a6fd51f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:31.366822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3bf73-6a3a-41b5-a830-149a6fd51f19 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:31.377732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1032/2060] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:31.394918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6616500-91f8-4a86-ae94-88a81f4087cd None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:31.440391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3bf73-6a3a-41b5-a830-149a6fd51f19 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:31.447856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-20b3bf73-6a3a-41b5-a830-149a6fd51f19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:31.475933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6616500-91f8-4a86-ae94-88a81f4087cd None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:31.481595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:31.482359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:31.482741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:31.483087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] PATH_INFO: `/v3/users/dcfe5d7a8aa549a3849f387fb5e3cdab` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:31.484684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcfe5d7a8aa549a3849f387fb5e3cdab)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:31.488181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:31.562566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3bf73-6a3a-41b5-a830-149a6fd51f19 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:31.568366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:31.569054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:31.569316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:31.569615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] PATH_INFO: `/v3/projects/858c5b2903e64efaa1d5c7ca0d29d471` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:31.570779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=858c5b2903e64efaa1d5c7ca0d29d471)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:31.573531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:31.619675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:31.656727 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcfe5d7a8aa549a3849f387fb5e3cdab'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.709013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '858c5b2903e64efaa1d5c7ca0d29d471', 'request_id': 'req-ac90bef1-eca9-49c6-9566-239697338f28'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.714774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '858c5b2903e64efaa1d5c7ca0d29d471', 'request_id': 'req-ac90bef1-eca9-49c6-9566-239697338f28'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.715570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcfe5d7a8aa549a3849f387fb5e3cdab'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.723365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcfe5d7a8aa549a3849f387fb5e3cdab'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.724213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcfe5d7a8aa549a3849f387fb5e3cdab'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.724829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f8040281-17da-493c-b498-6d7fef757ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcfe5d7a8aa549a3849f387fb5e3cdab'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.742611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1029/2061] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:31 2022] DELETE /identity/v3/users/dcfe5d7a8aa549a3849f387fb5e3cdab => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:31.749784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0d24f-9aea-43a2-9e32-96d0f7dc1196 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:31.749784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0d24f-9aea-43a2-9e32-96d0f7dc1196 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:31.749784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0d24f-9aea-43a2-9e32-96d0f7dc1196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:31.767576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1ab0d24f-9aea-43a2-9e32-96d0f7dc1196 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:31.772073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1ab0d24f-9aea-43a2-9e32-96d0f7dc1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:31.807384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '858c5b2903e64efaa1d5c7ca0d29d471', 'request_id': 'req-ac90bef1-eca9-49c6-9566-239697338f28'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.828975 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] The token cache is being invalidate because project 858c5b2903e64efaa1d5c7ca0d29d471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:31.829527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.830361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:31.835712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1033/2062] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:31 2022] DELETE /identity/v3/projects/858c5b2903e64efaa1d5c7ca0d29d471 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:31.839923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:31.841889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:31.848844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1030/2063] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:31.855670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0d24f-9aea-43a2-9e32-96d0f7dc1196 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:31.859488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1ab0d24f-9aea-43a2-9e32-96d0f7dc1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:31.912698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90bef1-eca9-49c6-9566-239697338f28 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:31.917456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:31.918101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:31.918658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:31.918658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] PATH_INFO: `/v3/users/fb2782f9e16a458b88e7dcdaec85b62e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:31.919962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb2782f9e16a458b88e7dcdaec85b62e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:31.921977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:31.929695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0d24f-9aea-43a2-9e32-96d0f7dc1196 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:31.933921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ccc0e-7171-495a-8d72-03158a87a401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:31.934527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ccc0e-7171-495a-8d72-03158a87a401 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:31.934802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ccc0e-7171-495a-8d72-03158a87a401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:31.935078 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ccc0e-7171-495a-8d72-03158a87a401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:31.947838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ccc0e-7171-495a-8d72-03158a87a401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:31.950783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ccc0e-7171-495a-8d72-03158a87a401 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:31.960001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1031/2064] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:32.154059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb2782f9e16a458b88e7dcdaec85b62e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.238721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb2782f9e16a458b88e7dcdaec85b62e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.263894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb2782f9e16a458b88e7dcdaec85b62e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.263894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb2782f9e16a458b88e7dcdaec85b62e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.263894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1fb7de7e-ee39-422f-8eb1-a4eaef1185bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb2782f9e16a458b88e7dcdaec85b62e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.291516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1034/2065] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:31 2022] DELETE /identity/v3/users/fb2782f9e16a458b88e7dcdaec85b62e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:32.512335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ccc0e-7171-495a-8d72-03158a87a401 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:32.514325 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-476ccc0e-7171-495a-8d72-03158a87a401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:32.541901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ccc0e-7171-495a-8d72-03158a87a401 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:32.657762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:32.660647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:32.660647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:32.660837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] PATH_INFO: `/v3/projects/cd3fadb8badc45f2a025ac771e294730` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:32.663126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3fadb8badc45f2a025ac771e294730)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:32.669118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:32.768425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:32.790606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3fadb8badc45f2a025ac771e294730', 'request_id': 'req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.797702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3fadb8badc45f2a025ac771e294730', 'request_id': 'req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.904458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3fadb8badc45f2a025ac771e294730', 'request_id': 'req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.921980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] The token cache is being invalidate because project cd3fadb8badc45f2a025ac771e294730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:32.922422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.923236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:32.925686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1032/2066] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:32 2022] DELETE /identity/v3/projects/cd3fadb8badc45f2a025ac771e294730 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:32.932098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7678e-0179-407b-a332-63ffded14686 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:32.943723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7678e-0179-407b-a332-63ffded14686 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:32.943723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7678e-0179-407b-a332-63ffded14686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:32.962117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7fa7678e-0179-407b-a332-63ffded14686 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:32.963720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7fa7678e-0179-407b-a332-63ffded14686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:32.972682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:32.974954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:33.074760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1035/2067] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:33.082694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191984c5-116a-4649-bafd-dd397736dbbb None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.083589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191984c5-116a-4649-bafd-dd397736dbbb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.083959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191984c5-116a-4649-bafd-dd397736dbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.085937 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef414d-53f9-4f06-9f81-71a04ebaf0e2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.091811 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.092492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.095345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.095345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] PATH_INFO: `/v3/projects/5fada3fb4a8b4dc6a223b78de4528e9b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.095345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fada3fb4a8b4dc6a223b78de4528e9b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:33.097987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:33.111483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-191984c5-116a-4649-bafd-dd397736dbbb None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:33.113525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-191984c5-116a-4649-bafd-dd397736dbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:33.144392 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1036/2068] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:33.157618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191984c5-116a-4649-bafd-dd397736dbbb None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:33.172588 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191984c5-116a-4649-bafd-dd397736dbbb None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.177775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6460c58-0ba0-4387-8315-9df83d1365ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.178402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6460c58-0ba0-4387-8315-9df83d1365ba admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.178735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6460c58-0ba0-4387-8315-9df83d1365ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.179012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6460c58-0ba0-4387-8315-9df83d1365ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.179665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6460c58-0ba0-4387-8315-9df83d1365ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:33.180008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6460c58-0ba0-4387-8315-9df83d1365ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:33.182171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6460c58-0ba0-4387-8315-9df83d1365ba admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:33.195564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1037/2069] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:10:33.209344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6460c58-0ba0-4387-8315-9df83d1365ba admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:33.222036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6460c58-0ba0-4387-8315-9df83d1365ba admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.230692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89385d8e-8d2a-4d95-8a08-3d44740b0ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.232719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89385d8e-8d2a-4d95-8a08-3d44740b0ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.233221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89385d8e-8d2a-4d95-8a08-3d44740b0ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.233896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89385d8e-8d2a-4d95-8a08-3d44740b0ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.234687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89385d8e-8d2a-4d95-8a08-3d44740b0ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:33.237432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89385d8e-8d2a-4d95-8a08-3d44740b0ae3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:33.257741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fada3fb4a8b4dc6a223b78de4528e9b', 'request_id': 'req-d93b4557-899c-45e9-8619-dd7eedab9186'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:33.271197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fada3fb4a8b4dc6a223b78de4528e9b', 'request_id': 'req-d93b4557-899c-45e9-8619-dd7eedab9186'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:33.366937 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fada3fb4a8b4dc6a223b78de4528e9b', 'request_id': 'req-d93b4557-899c-45e9-8619-dd7eedab9186'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:33.379874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1038/2070] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:33 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:33.381106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89385d8e-8d2a-4d95-8a08-3d44740b0ae3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:33.389823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] The token cache is being invalidate because project 5fada3fb4a8b4dc6a223b78de4528e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:33.390118 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:33.390820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d93b4557-899c-45e9-8619-dd7eedab9186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:33.392656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1033/2071] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:32 2022] DELETE /identity/v3/projects/5fada3fb4a8b4dc6a223b78de4528e9b => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:33.400568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0da6d-0352-4403-92e6-af5323c9d2cf None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.404161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0da6d-0352-4403-92e6-af5323c9d2cf None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.404161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0da6d-0352-4403-92e6-af5323c9d2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.425264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-eeb0da6d-0352-4403-92e6-af5323c9d2cf None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:33.426390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-eeb0da6d-0352-4403-92e6-af5323c9d2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:33.461530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89385d8e-8d2a-4d95-8a08-3d44740b0ae3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.470236 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24825f-3078-4502-ba4f-8c0b6ede4115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.470895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24825f-3078-4502-ba4f-8c0b6ede4115 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.471322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24825f-3078-4502-ba4f-8c0b6ede4115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.471808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24825f-3078-4502-ba4f-8c0b6ede4115 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.472446 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24825f-3078-4502-ba4f-8c0b6ede4115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:33.485928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24825f-3078-4502-ba4f-8c0b6ede4115 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:33.501656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1034/2072] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:33.503149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562e1db-46c3-4cb5-9db0-877c8658ef5e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.503486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562e1db-46c3-4cb5-9db0-877c8658ef5e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.503744 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562e1db-46c3-4cb5-9db0-877c8658ef5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.524991 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c562e1db-46c3-4cb5-9db0-877c8658ef5e None None] MFA Rules not processed for user `b7d7f6721a3a40f380578391949c443b`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:33.526953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c562e1db-46c3-4cb5-9db0-877c8658ef5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:33.585522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1035/2073] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:33.587061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fef191-7a8c-439f-a543-1787b154f832 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.587534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fef191-7a8c-439f-a543-1787b154f832 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.588070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fef191-7a8c-439f-a543-1787b154f832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.616007 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-43fef191-7a8c-439f-a543-1787b154f832 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:33.617573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-43fef191-7a8c-439f-a543-1787b154f832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:33.627853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1036/2074] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:33.636704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fef191-7a8c-439f-a543-1787b154f832 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:33.639267 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1039/2075] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:33 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:33.647901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fef191-7a8c-439f-a543-1787b154f832 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.648769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24825f-3078-4502-ba4f-8c0b6ede4115 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:33.651837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2f24825f-3078-4502-ba4f-8c0b6ede4115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:33.665228 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d12d06-ef3f-42f5-96a6-9a5f7d1930ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.666243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d12d06-ef3f-42f5-96a6-9a5f7d1930ef admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.666514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d12d06-ef3f-42f5-96a6-9a5f7d1930ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.666828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d12d06-ef3f-42f5-96a6-9a5f7d1930ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.667442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d12d06-ef3f-42f5-96a6-9a5f7d1930ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:33.667738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d12d06-ef3f-42f5-96a6-9a5f7d1930ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:33.669522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d12d06-ef3f-42f5-96a6-9a5f7d1930ef admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:33.686067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24825f-3078-4502-ba4f-8c0b6ede4115 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.692565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1037/2076] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:33.692717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0916e-a0c2-4269-97cd-60d02cac1063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.692717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0916e-a0c2-4269-97cd-60d02cac1063 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.693198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0916e-a0c2-4269-97cd-60d02cac1063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.693534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0916e-a0c2-4269-97cd-60d02cac1063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.694247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0916e-a0c2-4269-97cd-60d02cac1063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:33.695278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d12d06-ef3f-42f5-96a6-9a5f7d1930ef admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:33.696011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0916e-a0c2-4269-97cd-60d02cac1063 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:33.706545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d12d06-ef3f-42f5-96a6-9a5f7d1930ef admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.708191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1040/2077] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:33.711698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4d7a4-8181-42f0-a465-bf94ee785290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.712282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4d7a4-8181-42f0-a465-bf94ee785290 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.712412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0916e-a0c2-4269-97cd-60d02cac1063 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:33.712618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4d7a4-8181-42f0-a465-bf94ee785290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.713022 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4d7a4-8181-42f0-a465-bf94ee785290 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.714170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4d7a4-8181-42f0-a465-bf94ee785290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:33.716120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4d7a4-8181-42f0-a465-bf94ee785290 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:33.731791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0916e-a0c2-4269-97cd-60d02cac1063 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.738754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f188625-8541-405b-b6ca-34c8472fe398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.739485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f188625-8541-405b-b6ca-34c8472fe398 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.739760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f188625-8541-405b-b6ca-34c8472fe398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.740074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f188625-8541-405b-b6ca-34c8472fe398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.740772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f188625-8541-405b-b6ca-34c8472fe398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:33.742959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f188625-8541-405b-b6ca-34c8472fe398 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:33.758991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1041/2078] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:33.763782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f188625-8541-405b-b6ca-34c8472fe398 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:33.775413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f188625-8541-405b-b6ca-34c8472fe398 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.783152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330d6f6-2ef6-47c4-8b3b-dc934a7033bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.784343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330d6f6-2ef6-47c4-8b3b-dc934a7033bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.784817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330d6f6-2ef6-47c4-8b3b-dc934a7033bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.785229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330d6f6-2ef6-47c4-8b3b-dc934a7033bf admin admin] PATH_INFO: `/v3/projects/d4d9af0e1c65440f816a850317372e1b/users/92119b2942df4136a5d3d436d2c54446/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.802689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330d6f6-2ef6-47c4-8b3b-dc934a7033bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d9af0e1c65440f816a850317372e1b, user_id=92119b2942df4136a5d3d436d2c54446, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:33.814952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:33.818531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330d6f6-2ef6-47c4-8b3b-dc934a7033bf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:33.820629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1038/2079] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:33 2022] POST /identity/v3/projects => generated 399 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:33.825196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4d7a4-8181-42f0-a465-bf94ee785290 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:33.912363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1042/2080] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:33 2022] PUT /identity/v3/projects/d4d9af0e1c65440f816a850317372e1b/users/92119b2942df4136a5d3d436d2c54446/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:33.922899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168a682-3e4f-419c-b2bd-c6192cfc2b21 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.923177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168a682-3e4f-419c-b2bd-c6192cfc2b21 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.923461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168a682-3e4f-419c-b2bd-c6192cfc2b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.924013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4d7a4-8181-42f0-a465-bf94ee785290 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:33.977550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3168a682-3e4f-419c-b2bd-c6192cfc2b21 None None] MFA Rules not processed for user `92119b2942df4136a5d3d436d2c54446`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:33.978856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3168a682-3e4f-419c-b2bd-c6192cfc2b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:33.996566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ca52-d19f-437b-b20f-281afd1b229f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:33.997222 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ca52-d19f-437b-b20f-281afd1b229f admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:33.997511 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ca52-d19f-437b-b20f-281afd1b229f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:33.997746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6ca52-d19f-437b-b20f-281afd1b229f admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:33.998365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6ca52-d19f-437b-b20f-281afd1b229f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:34.000543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6ca52-d19f-437b-b20f-281afd1b229f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:34.100608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1043/2081] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:34.112711 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168a682-3e4f-419c-b2bd-c6192cfc2b21 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:34.114318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3168a682-3e4f-419c-b2bd-c6192cfc2b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:34.127836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1039/2082] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:33 2022] POST /identity/v3/users => generated 300 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:34.138458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ca52-d19f-437b-b20f-281afd1b229f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:34.149554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6ca52-d19f-437b-b20f-281afd1b229f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:34.165940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e980da-a493-4e9d-a100-fe7ca5480882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:34.167701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e980da-a493-4e9d-a100-fe7ca5480882 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:34.168110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e980da-a493-4e9d-a100-fe7ca5480882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:34.168515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e980da-a493-4e9d-a100-fe7ca5480882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:34.169844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e980da-a493-4e9d-a100-fe7ca5480882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:34.186449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e980da-a493-4e9d-a100-fe7ca5480882 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:34.204493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1040/2083] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:34.208732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e980da-a493-4e9d-a100-fe7ca5480882 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:34.227210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e980da-a493-4e9d-a100-fe7ca5480882 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:34.235579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9777033-d22c-4317-abbf-e9459f54b2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:34.236185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9777033-d22c-4317-abbf-e9459f54b2df admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:34.249431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9777033-d22c-4317-abbf-e9459f54b2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:34.250085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9777033-d22c-4317-abbf-e9459f54b2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:34.251962 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:34.253605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9777033-d22c-4317-abbf-e9459f54b2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:34.253718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168a682-3e4f-419c-b2bd-c6192cfc2b21 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:34.257306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9777033-d22c-4317-abbf-e9459f54b2df admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:34.257798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0b6d9-0769-4141-8aa9-8d719a19220a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:34.258586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0b6d9-0769-4141-8aa9-8d719a19220a service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:34.259024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0b6d9-0769-4141-8aa9-8d719a19220a service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:34.259200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0b6d9-0769-4141-8aa9-8d719a19220a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:34.271844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0b6d9-0769-4141-8aa9-8d719a19220a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:34.273748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0b6d9-0769-4141-8aa9-8d719a19220a service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:34.294802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1044/2084] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:34.298359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1041/2085] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:34.302650 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0b6d9-0769-4141-8aa9-8d719a19220a service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:34.317036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0b6d9-0769-4141-8aa9-8d719a19220a service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:34.323813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf68311-56af-4adf-ac8c-e958ddc1a1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:34.324956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf68311-56af-4adf-ac8c-e958ddc1a1ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:34.325257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf68311-56af-4adf-ac8c-e958ddc1a1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:34.325516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf68311-56af-4adf-ac8c-e958ddc1a1ce admin admin] PATH_INFO: `/v3/projects/0a6ca8879a8d4944a25e9cc897cc6202/users/09960b0a84c04f229da37505a7425102/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:34.352268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf68311-56af-4adf-ac8c-e958ddc1a1ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6ca8879a8d4944a25e9cc897cc6202, user_id=09960b0a84c04f229da37505a7425102, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:34.355274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf68311-56af-4adf-ac8c-e958ddc1a1ce admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:34.459174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1045/2086] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:34 2022] PUT /identity/v3/projects/0a6ca8879a8d4944a25e9cc897cc6202/users/09960b0a84c04f229da37505a7425102/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:34.469972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b905ab-a922-48a3-b147-b95de9b00954 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:34.470291 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b905ab-a922-48a3-b147-b95de9b00954 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:34.470541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b905ab-a922-48a3-b147-b95de9b00954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:34.504439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-96b905ab-a922-48a3-b147-b95de9b00954 None None] MFA Rules not processed for user `09960b0a84c04f229da37505a7425102`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:34.505726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-96b905ab-a922-48a3-b147-b95de9b00954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:34.583080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1042/2087] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:34.588132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf68311-56af-4adf-ac8c-e958ddc1a1ce admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:34.660038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf68311-56af-4adf-ac8c-e958ddc1a1ce admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:34.667233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e4432-2908-4923-9eb6-aa018213757a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:34.667233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e4432-2908-4923-9eb6-aa018213757a admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:34.667233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e4432-2908-4923-9eb6-aa018213757a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:34.667233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e4432-2908-4923-9eb6-aa018213757a admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:34.667624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e4432-2908-4923-9eb6-aa018213757a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:34.669164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e4432-2908-4923-9eb6-aa018213757a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:34.779833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1046/2088] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:34 2022] POST /identity/v3/projects => generated 401 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:34.783891 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b905ab-a922-48a3-b147-b95de9b00954 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:34.795322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b905ab-a922-48a3-b147-b95de9b00954 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:34.873437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc84f29-9ad4-4f68-9495-3d25e74c8292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:34.876543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc84f29-9ad4-4f68-9495-3d25e74c8292 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:34.876543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc84f29-9ad4-4f68-9495-3d25e74c8292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:34.876543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc84f29-9ad4-4f68-9495-3d25e74c8292 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:34.876543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc84f29-9ad4-4f68-9495-3d25e74c8292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:34.877200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc84f29-9ad4-4f68-9495-3d25e74c8292 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:34.950200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1043/2089] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:34 2022] POST /identity/v3/users => generated 307 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:34.954095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e4432-2908-4923-9eb6-aa018213757a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:34.964318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e4432-2908-4923-9eb6-aa018213757a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:34.969442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bee8d-9911-4669-a153-0b75be587a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:34.969964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bee8d-9911-4669-a153-0b75be587a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:34.970243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bee8d-9911-4669-a153-0b75be587a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:34.970503 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bee8d-9911-4669-a153-0b75be587a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:34.971490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66bee8d-9911-4669-a153-0b75be587a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:34.973104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66bee8d-9911-4669-a153-0b75be587a7d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:34.984906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1047/2090] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:34.989273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc84f29-9ad4-4f68-9495-3d25e74c8292 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:35.001587 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc84f29-9ad4-4f68-9495-3d25e74c8292 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:35.006544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af858ba8-6921-4f13-8aef-d08d3cc2c7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:35.007341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af858ba8-6921-4f13-8aef-d08d3cc2c7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:35.007450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af858ba8-6921-4f13-8aef-d08d3cc2c7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:35.007719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af858ba8-6921-4f13-8aef-d08d3cc2c7b9 admin admin] PATH_INFO: `/v3/projects/1a14eb6130f34ee199ea976ac469f82c/users/b1f27e7d730b4ebdba76f672f34ec57d/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:35.024153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af858ba8-6921-4f13-8aef-d08d3cc2c7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a14eb6130f34ee199ea976ac469f82c, user_id=b1f27e7d730b4ebdba76f672f34ec57d, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:35.027574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af858ba8-6921-4f13-8aef-d08d3cc2c7b9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:35.139502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1044/2091] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:34 2022] PUT /identity/v3/projects/1a14eb6130f34ee199ea976ac469f82c/users/b1f27e7d730b4ebdba76f672f34ec57d/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:35.140756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bee8d-9911-4669-a153-0b75be587a7d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:35.155953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bee8d-9911-4669-a153-0b75be587a7d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:35.215619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6035ae-2eb9-4d73-8987-232a4aa0dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:35.216230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6035ae-2eb9-4d73-8987-232a4aa0dc07 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:35.216522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6035ae-2eb9-4d73-8987-232a4aa0dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:35.216802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6035ae-2eb9-4d73-8987-232a4aa0dc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:35.217437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6035ae-2eb9-4d73-8987-232a4aa0dc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:35.219074 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6035ae-2eb9-4d73-8987-232a4aa0dc07 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:35.232949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1048/2092] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:35.234173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af858ba8-6921-4f13-8aef-d08d3cc2c7b9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:35.246878 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af858ba8-6921-4f13-8aef-d08d3cc2c7b9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:35.253781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1691ee1-1a8b-4ed1-8cb3-f11f9783d212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:35.254220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1691ee1-1a8b-4ed1-8cb3-f11f9783d212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:35.254820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1691ee1-1a8b-4ed1-8cb3-f11f9783d212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:35.255145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1691ee1-1a8b-4ed1-8cb3-f11f9783d212 admin admin] PATH_INFO: `/v3/projects/1a14eb6130f34ee199ea976ac469f82c/users/b1f27e7d730b4ebdba76f672f34ec57d/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:35.257367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1691ee1-1a8b-4ed1-8cb3-f11f9783d212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a14eb6130f34ee199ea976ac469f82c, user_id=b1f27e7d730b4ebdba76f672f34ec57d, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:35.261101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1691ee1-1a8b-4ed1-8cb3-f11f9783d212 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:35.275859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1045/2093] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:35 2022] PUT /identity/v3/projects/1a14eb6130f34ee199ea976ac469f82c/users/b1f27e7d730b4ebdba76f672f34ec57d/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:35.287883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66c531-df3a-4a95-bfda-e961d84269be None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:35.288351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66c531-df3a-4a95-bfda-e961d84269be None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:35.288689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66c531-df3a-4a95-bfda-e961d84269be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:35.321780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2c66c531-df3a-4a95-bfda-e961d84269be None None] MFA Rules not processed for user `b1f27e7d730b4ebdba76f672f34ec57d`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:35.323160 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2c66c531-df3a-4a95-bfda-e961d84269be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:35.415235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1049/2094] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:35 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:35.427517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1691ee1-1a8b-4ed1-8cb3-f11f9783d212 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:35.486284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1691ee1-1a8b-4ed1-8cb3-f11f9783d212 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:35.490322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd3ad7-1ff9-44dd-aaef-ea2025ea894d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:35.491081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd3ad7-1ff9-44dd-aaef-ea2025ea894d service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:35.491189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd3ad7-1ff9-44dd-aaef-ea2025ea894d service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:35.491423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd3ad7-1ff9-44dd-aaef-ea2025ea894d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:35.501161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd3ad7-1ff9-44dd-aaef-ea2025ea894d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:35.502580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd3ad7-1ff9-44dd-aaef-ea2025ea894d service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:35.512084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1046/2095] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:35 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:35.747078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66c531-df3a-4a95-bfda-e961d84269be None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:35.758937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66c531-df3a-4a95-bfda-e961d84269be None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:35.767369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44470a-1199-45c4-bc5c-20651418982b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:35.770788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44470a-1199-45c4-bc5c-20651418982b service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:35.770788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44470a-1199-45c4-bc5c-20651418982b service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:35.770788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44470a-1199-45c4-bc5c-20651418982b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:35.864614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44470a-1199-45c4-bc5c-20651418982b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:35.866110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44470a-1199-45c4-bc5c-20651418982b service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:35.881278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1050/2096] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:35 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:36.115896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:36.557742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:36.689166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:37.070953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd3ad7-1ff9-44dd-aaef-ea2025ea894d service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:37.072448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0cbd3ad7-1ff9-44dd-aaef-ea2025ea894d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:37.138559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd3ad7-1ff9-44dd-aaef-ea2025ea894d service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:37.143159 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:37.143770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:37.144050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:37.144321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] PATH_INFO: `/v3/users/288aee37d37e49c0a4600e708e9ac3b6` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:37.145477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=288aee37d37e49c0a4600e708e9ac3b6)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:37.147394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:37.374697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '288aee37d37e49c0a4600e708e9ac3b6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:37.417872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '288aee37d37e49c0a4600e708e9ac3b6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:37.424806 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '288aee37d37e49c0a4600e708e9ac3b6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:37.425428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '288aee37d37e49c0a4600e708e9ac3b6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:37.426124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '288aee37d37e49c0a4600e708e9ac3b6'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:37.442788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1047/2097] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:37 2022] DELETE /identity/v3/users/288aee37d37e49c0a4600e708e9ac3b6 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:37.722981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44470a-1199-45c4-bc5c-20651418982b service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:37.741005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44470a-1199-45c4-bc5c-20651418982b service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:37.802923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:37.803714 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:37.803995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:37.804316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] PATH_INFO: `/v3/projects/3b876704aecf424ab54cd836d65aadbc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:37.805650 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b876704aecf424ab54cd836d65aadbc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:37.807606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:37.852576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:37.941859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b876704aecf424ab54cd836d65aadbc', 'request_id': 'req-e0052a91-5855-43fd-a73f-e0b6db2c8035'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:37.949295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b876704aecf424ab54cd836d65aadbc', 'request_id': 'req-e0052a91-5855-43fd-a73f-e0b6db2c8035'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.023792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b876704aecf424ab54cd836d65aadbc', 'request_id': 'req-e0052a91-5855-43fd-a73f-e0b6db2c8035'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.039460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] The token cache is being invalidate because project 3b876704aecf424ab54cd836d65aadbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:38.039750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.052465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.054459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1051/2098] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:37 2022] DELETE /identity/v3/projects/3b876704aecf424ab54cd836d65aadbc => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:38.058234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:38.059719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:38.074569 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a5c05-2030-4ede-88d8-18131f318a22 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:38.135817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:38.136653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:38.136936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:38.137210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] PATH_INFO: `/v3/users/7fdc81dc9eb14461bcf544d2a417dc9f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:38.138409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fdc81dc9eb14461bcf544d2a417dc9f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:38.140490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:38.355774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:38.373406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fdc81dc9eb14461bcf544d2a417dc9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.440631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fdc81dc9eb14461bcf544d2a417dc9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.449581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fdc81dc9eb14461bcf544d2a417dc9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.450228 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fdc81dc9eb14461bcf544d2a417dc9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.451059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-194c36fb-59f0-4bf1-8268-078edc38fa9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fdc81dc9eb14461bcf544d2a417dc9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.470641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1048/2099] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:38 2022] DELETE /identity/v3/users/7fdc81dc9eb14461bcf544d2a417dc9f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:38.740252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:38.741930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:38.763617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0052a91-5855-43fd-a73f-e0b6db2c8035 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:38.819486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:38.830064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:38.830686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:38.830963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:38.831235 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] PATH_INFO: `/v3/projects/4d3053f55feb4a8abf1234688d448f0d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:38.832418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d3053f55feb4a8abf1234688d448f0d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:38.841655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:38.938015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d3053f55feb4a8abf1234688d448f0d', 'request_id': 'req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:38.943345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d3053f55feb4a8abf1234688d448f0d', 'request_id': 'req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:39.006343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d3053f55feb4a8abf1234688d448f0d', 'request_id': 'req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:39.018549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] The token cache is being invalidate because project 4d3053f55feb4a8abf1234688d448f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:39.018834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:39.019457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cf9ecc9d-3e86-4450-96b9-45a068fc14f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:39.020982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1052/2100] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:38 2022] DELETE /identity/v3/projects/4d3053f55feb4a8abf1234688d448f0d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:39.026081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395488fb-6679-4ca3-9e3b-49b91b8b4642 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.026368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395488fb-6679-4ca3-9e3b-49b91b8b4642 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.026683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395488fb-6679-4ca3-9e3b-49b91b8b4642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.046699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-395488fb-6679-4ca3-9e3b-49b91b8b4642 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:39.047896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-395488fb-6679-4ca3-9e3b-49b91b8b4642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:39.105240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1049/2101] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:39.113640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867625b4-5d50-483b-b099-aa0e968d44ca None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.113943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867625b4-5d50-483b-b099-aa0e968d44ca None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.114211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867625b4-5d50-483b-b099-aa0e968d44ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.137158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-867625b4-5d50-483b-b099-aa0e968d44ca None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:39.138643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-867625b4-5d50-483b-b099-aa0e968d44ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:39.153040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1053/2102] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:39.157681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395488fb-6679-4ca3-9e3b-49b91b8b4642 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:39.168619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395488fb-6679-4ca3-9e3b-49b91b8b4642 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:39.173698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe7e24-5a5a-460b-8372-ed049d8563c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:39.174615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe7e24-5a5a-460b-8372-ed049d8563c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.174890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe7e24-5a5a-460b-8372-ed049d8563c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.175156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe7e24-5a5a-460b-8372-ed049d8563c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.175817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe7e24-5a5a-460b-8372-ed049d8563c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:39.176110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe7e24-5a5a-460b-8372-ed049d8563c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:39.177789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe7e24-5a5a-460b-8372-ed049d8563c5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:39.191317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1050/2103] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:39.195849 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867625b4-5d50-483b-b099-aa0e968d44ca None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:39.206995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867625b4-5d50-483b-b099-aa0e968d44ca None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:39.212191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79f35a-c2c3-4a76-b926-049580686e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:39.213348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79f35a-c2c3-4a76-b926-049580686e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.213674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79f35a-c2c3-4a76-b926-049580686e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.213939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79f35a-c2c3-4a76-b926-049580686e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.214587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79f35a-c2c3-4a76-b926-049580686e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:39.216828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79f35a-c2c3-4a76-b926-049580686e35 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:39.235943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:39.286145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1054/2104] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:39 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:39.287305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe7e24-5a5a-460b-8372-ed049d8563c5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:39.304643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe7e24-5a5a-460b-8372-ed049d8563c5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:39.390198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e53833-7a8e-4db2-b2a7-79e598b1e4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:39.390957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e53833-7a8e-4db2-b2a7-79e598b1e4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.391238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e53833-7a8e-4db2-b2a7-79e598b1e4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.391504 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e53833-7a8e-4db2-b2a7-79e598b1e4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.392134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e53833-7a8e-4db2-b2a7-79e598b1e4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:39.394331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e53833-7a8e-4db2-b2a7-79e598b1e4c6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:39.465566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1051/2105] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:39 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:39.469611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79f35a-c2c3-4a76-b926-049580686e35 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:39.480049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79f35a-c2c3-4a76-b926-049580686e35 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:39.484802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94b616-c463-4561-96e3-a461f0bb4f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:39.485390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b616-c463-4561-96e3-a461f0bb4f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.485665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b616-c463-4561-96e3-a461f0bb4f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.485935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b616-c463-4561-96e3-a461f0bb4f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.486569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94b616-c463-4561-96e3-a461f0bb4f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:39.488168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94b616-c463-4561-96e3-a461f0bb4f7e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:39.497989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1055/2106] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:39.502450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e53833-7a8e-4db2-b2a7-79e598b1e4c6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:39.503926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:39.512842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e53833-7a8e-4db2-b2a7-79e598b1e4c6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:39.517458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e2d1a-9ffa-4170-a199-846e507cabc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:39.518040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e2d1a-9ffa-4170-a199-846e507cabc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.518313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e2d1a-9ffa-4170-a199-846e507cabc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.518577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e2d1a-9ffa-4170-a199-846e507cabc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.519205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e2d1a-9ffa-4170-a199-846e507cabc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:39.520883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e2d1a-9ffa-4170-a199-846e507cabc7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:39.535579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1052/2107] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:39.536773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94b616-c463-4561-96e3-a461f0bb4f7e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:39.548549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94b616-c463-4561-96e3-a461f0bb4f7e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:39.553237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dede96-5389-40d9-93a1-8e948fd7b4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:39.553883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dede96-5389-40d9-93a1-8e948fd7b4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.554157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dede96-5389-40d9-93a1-8e948fd7b4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.554422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dede96-5389-40d9-93a1-8e948fd7b4b4 admin admin] PATH_INFO: `/v3/projects/72d20e1627fa4f72857d077c2bc15c33/users/3bb8dfebe63944f395d17ba532b807a3/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.568890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dede96-5389-40d9-93a1-8e948fd7b4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d20e1627fa4f72857d077c2bc15c33, user_id=3bb8dfebe63944f395d17ba532b807a3, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:39.572025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dede96-5389-40d9-93a1-8e948fd7b4b4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:39.601522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:39.631332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1056/2108] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:39 2022] PUT /identity/v3/projects/72d20e1627fa4f72857d077c2bc15c33/users/3bb8dfebe63944f395d17ba532b807a3/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:39.642298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af920506-d224-4244-b5a0-aa5aa4b9afc2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.647691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af920506-d224-4244-b5a0-aa5aa4b9afc2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.647691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af920506-d224-4244-b5a0-aa5aa4b9afc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.671924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-af920506-d224-4244-b5a0-aa5aa4b9afc2 None None] MFA Rules not processed for user `3bb8dfebe63944f395d17ba532b807a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:39.673150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-af920506-d224-4244-b5a0-aa5aa4b9afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:39.738513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1053/2109] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:39.750030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dede96-5389-40d9-93a1-8e948fd7b4b4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:39.751723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-74dede96-5389-40d9-93a1-8e948fd7b4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:39.838621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dede96-5389-40d9-93a1-8e948fd7b4b4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:39.839451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d6ae2-eb41-4685-a587-d3df9443cda2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e95d295337b4472be946d252ac3bd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:39.840379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d6ae2-eb41-4685-a587-d3df9443cda2 service glance] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:39.840719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d6ae2-eb41-4685-a587-d3df9443cda2 service glance] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:39.840964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d6ae2-eb41-4685-a587-d3df9443cda2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:39.852909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d6ae2-eb41-4685-a587-d3df9443cda2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:39.854370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d6ae2-eb41-4685-a587-d3df9443cda2 service glance] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:39.863534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1057/2110] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:39.865127 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:39.933102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:39.955572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:40.052140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:40.138366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:40.407090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:40.630511 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:41.033740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af920506-d224-4244-b5a0-aa5aa4b9afc2 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:41.035189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-af920506-d224-4244-b5a0-aa5aa4b9afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:41.103199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af920506-d224-4244-b5a0-aa5aa4b9afc2 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:41.108590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:41.109414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:41.112723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:41.112723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] PATH_INFO: `/v3/users/09960b0a84c04f229da37505a7425102` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:41.112723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09960b0a84c04f229da37505a7425102)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:41.113620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:41.396439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:41.399222 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09960b0a84c04f229da37505a7425102'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:41.447259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09960b0a84c04f229da37505a7425102'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:41.456604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09960b0a84c04f229da37505a7425102'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:41.457407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09960b0a84c04f229da37505a7425102'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:41.458062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09960b0a84c04f229da37505a7425102'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:41.479835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1054/2111] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:41 2022] DELETE /identity/v3/users/09960b0a84c04f229da37505a7425102 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:41.484617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae0d94-3972-49b2-8a2d-99b92536d5bb None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:41.485078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae0d94-3972-49b2-8a2d-99b92536d5bb None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:41.485381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae0d94-3972-49b2-8a2d-99b92536d5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:41.505473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a4ae0d94-3972-49b2-8a2d-99b92536d5bb None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:41.506671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a4ae0d94-3972-49b2-8a2d-99b92536d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:41.566332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1058/2112] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:41.577345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:41.581539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:41.645268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5516fc-bd2e-4a20-b0e1-15831bcacab8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:41.648874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5269852-1eaa-45b4-bcd2-5ed2450a7cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:41.649470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5269852-1eaa-45b4-bcd2-5ed2450a7cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:41.649778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5269852-1eaa-45b4-bcd2-5ed2450a7cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:41.650048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5269852-1eaa-45b4-bcd2-5ed2450a7cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:41.661657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5269852-1eaa-45b4-bcd2-5ed2450a7cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:41.663095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5269852-1eaa-45b4-bcd2-5ed2450a7cb4 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:41.673629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1055/2113] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:42.140342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae0d94-3972-49b2-8a2d-99b92536d5bb None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:42.142168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a4ae0d94-3972-49b2-8a2d-99b92536d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:42.155221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae0d94-3972-49b2-8a2d-99b92536d5bb None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:42.163863 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:42.164730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:42.165212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:42.165371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] PATH_INFO: `/v3/projects/0a6ca8879a8d4944a25e9cc897cc6202` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:42.166657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6ca8879a8d4944a25e9cc897cc6202)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:42.168530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:42.176698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:42.293728 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6ca8879a8d4944a25e9cc897cc6202', 'request_id': 'req-051e1453-5abf-4019-83ab-ecb9d22c6455'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.299948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6ca8879a8d4944a25e9cc897cc6202', 'request_id': 'req-051e1453-5abf-4019-83ab-ecb9d22c6455'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.383679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6ca8879a8d4944a25e9cc897cc6202', 'request_id': 'req-051e1453-5abf-4019-83ab-ecb9d22c6455'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.397860 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] The token cache is being invalidate because project 0a6ca8879a8d4944a25e9cc897cc6202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:42.398144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.398861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.400356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1059/2114] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:42 2022] DELETE /identity/v3/projects/0a6ca8879a8d4944a25e9cc897cc6202 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:42.404299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5269852-1eaa-45b4-bcd2-5ed2450a7cb4 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:42.414040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5269852-1eaa-45b4-bcd2-5ed2450a7cb4 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:42.479453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:42.480149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:42.480430 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:42.480768 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] PATH_INFO: `/v3/users/b1f27e7d730b4ebdba76f672f34ec57d` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:42.481899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f27e7d730b4ebdba76f672f34ec57d)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:42.483914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:42.689420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f27e7d730b4ebdba76f672f34ec57d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.760999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f27e7d730b4ebdba76f672f34ec57d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.768294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f27e7d730b4ebdba76f672f34ec57d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.768936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f27e7d730b4ebdba76f672f34ec57d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.769533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-a126d037-9992-42e8-865e-8bb3890a8cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f27e7d730b4ebdba76f672f34ec57d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:42.771469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:42.786587 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1056/2115] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:42 2022] DELETE /identity/v3/users/b1f27e7d730b4ebdba76f672f34ec57d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:43.265370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:43.266920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:43.280719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e1453-5abf-4019-83ab-ecb9d22c6455 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:43.342489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:43.343148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:43.343419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:43.343685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] PATH_INFO: `/v3/projects/1a14eb6130f34ee199ea976ac469f82c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:43.345157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a14eb6130f34ee199ea976ac469f82c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:43.347149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:43.467729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a14eb6130f34ee199ea976ac469f82c', 'request_id': 'req-b228fc7c-6084-46ff-86fe-6c0a62e61587'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:43.474164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a14eb6130f34ee199ea976ac469f82c', 'request_id': 'req-b228fc7c-6084-46ff-86fe-6c0a62e61587'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:43.532302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a14eb6130f34ee199ea976ac469f82c', 'request_id': 'req-b228fc7c-6084-46ff-86fe-6c0a62e61587'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:43.546817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] The token cache is being invalidate because project 1a14eb6130f34ee199ea976ac469f82c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:43.547173 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:43.548495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b228fc7c-6084-46ff-86fe-6c0a62e61587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:43.554137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1060/2116] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:43 2022] DELETE /identity/v3/projects/1a14eb6130f34ee199ea976ac469f82c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:43.555966 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf149b-e9a8-4d31-8652-9368b85ea707 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:43.556954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf149b-e9a8-4d31-8652-9368b85ea707 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:43.557229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf149b-e9a8-4d31-8652-9368b85ea707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:43.577853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-dbaf149b-e9a8-4d31-8652-9368b85ea707 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:43.579069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-dbaf149b-e9a8-4d31-8652-9368b85ea707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:43.640646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1057/2117] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:43.648094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8411b3-801f-46a0-91f1-f53cf99bc19e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:43.648367 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8411b3-801f-46a0-91f1-f53cf99bc19e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:43.648736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8411b3-801f-46a0-91f1-f53cf99bc19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:43.668816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4f8411b3-801f-46a0-91f1-f53cf99bc19e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:43.669998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4f8411b3-801f-46a0-91f1-f53cf99bc19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:43.677812 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1061/2118] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:43.682109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf149b-e9a8-4d31-8652-9368b85ea707 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:43.692158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf149b-e9a8-4d31-8652-9368b85ea707 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:43.696980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd9e49-b6b0-472e-a3f8-0abb5f64041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:43.697571 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd9e49-b6b0-472e-a3f8-0abb5f64041f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:43.697846 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd9e49-b6b0-472e-a3f8-0abb5f64041f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:43.698115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd9e49-b6b0-472e-a3f8-0abb5f64041f admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:43.699026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd9e49-b6b0-472e-a3f8-0abb5f64041f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:43.699437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd9e49-b6b0-472e-a3f8-0abb5f64041f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:43.701041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd9e49-b6b0-472e-a3f8-0abb5f64041f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:43.713042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1058/2119] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:43.717919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8411b3-801f-46a0-91f1-f53cf99bc19e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:43.741604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8411b3-801f-46a0-91f1-f53cf99bc19e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:43.748761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6bdd4-c316-49fd-8344-8d80c87b12db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:43.749524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6bdd4-c316-49fd-8344-8d80c87b12db admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:43.749972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6bdd4-c316-49fd-8344-8d80c87b12db admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:43.750246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6bdd4-c316-49fd-8344-8d80c87b12db admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:43.750939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6bdd4-c316-49fd-8344-8d80c87b12db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:43.753015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6bdd4-c316-49fd-8344-8d80c87b12db admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:43.771190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:43.837466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1062/2120] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:43 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:43.841672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd9e49-b6b0-472e-a3f8-0abb5f64041f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:43.851792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd9e49-b6b0-472e-a3f8-0abb5f64041f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:43.911339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac5789-a723-442e-9ddf-c85ecb6384dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:43.911948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac5789-a723-442e-9ddf-c85ecb6384dc admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:43.912227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac5789-a723-442e-9ddf-c85ecb6384dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:43.912497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac5789-a723-442e-9ddf-c85ecb6384dc admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:43.913169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac5789-a723-442e-9ddf-c85ecb6384dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:43.915612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac5789-a723-442e-9ddf-c85ecb6384dc admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:43.998400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1059/2121] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:43 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:43.999644 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6bdd4-c316-49fd-8344-8d80c87b12db admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:44.012174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6bdd4-c316-49fd-8344-8d80c87b12db admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:44.019030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acd2bc-da7d-4f49-9ed8-9177918c1e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:44.019627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acd2bc-da7d-4f49-9ed8-9177918c1e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.019927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acd2bc-da7d-4f49-9ed8-9177918c1e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.020187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acd2bc-da7d-4f49-9ed8-9177918c1e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.020888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acd2bc-da7d-4f49-9ed8-9177918c1e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:44.022523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acd2bc-da7d-4f49-9ed8-9177918c1e2c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:44.036557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1063/2122] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:44.039386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac5789-a723-442e-9ddf-c85ecb6384dc admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:44.050267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac5789-a723-442e-9ddf-c85ecb6384dc admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:44.055341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8123fa3-9484-4c09-a2d2-bd5e0d6f2936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:44.056527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8123fa3-9484-4c09-a2d2-bd5e0d6f2936 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.056952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8123fa3-9484-4c09-a2d2-bd5e0d6f2936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.057848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8123fa3-9484-4c09-a2d2-bd5e0d6f2936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.057969 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8123fa3-9484-4c09-a2d2-bd5e0d6f2936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:44.059532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8123fa3-9484-4c09-a2d2-bd5e0d6f2936 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:44.070841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1060/2123] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:44.080097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acd2bc-da7d-4f49-9ed8-9177918c1e2c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:44.091253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acd2bc-da7d-4f49-9ed8-9177918c1e2c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:44.097431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18173b10-732d-4c7c-abad-1b2d853bc3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:44.101878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18173b10-732d-4c7c-abad-1b2d853bc3c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.101878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18173b10-732d-4c7c-abad-1b2d853bc3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.101878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18173b10-732d-4c7c-abad-1b2d853bc3c4 admin admin] PATH_INFO: `/v3/projects/c976e4ae291343b6a1959563d936df2d/users/839dd931ca314062a1de67ca6b7d9bb6/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.121404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18173b10-732d-4c7c-abad-1b2d853bc3c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c976e4ae291343b6a1959563d936df2d, user_id=839dd931ca314062a1de67ca6b7d9bb6, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:44.124377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18173b10-732d-4c7c-abad-1b2d853bc3c4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:44.207470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1064/2124] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:44 2022] PUT /identity/v3/projects/c976e4ae291343b6a1959563d936df2d/users/839dd931ca314062a1de67ca6b7d9bb6/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:44.218799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30d679-df04-4bd4-8676-e9ca1061d70e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.219163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30d679-df04-4bd4-8676-e9ca1061d70e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.219498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30d679-df04-4bd4-8676-e9ca1061d70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.251668 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1f30d679-df04-4bd4-8676-e9ca1061d70e None None] MFA Rules not processed for user `839dd931ca314062a1de67ca6b7d9bb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:44.253212 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1f30d679-df04-4bd4-8676-e9ca1061d70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:44.295986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:44.321932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1061/2125] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:44.326817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18173b10-732d-4c7c-abad-1b2d853bc3c4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:44.393292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18173b10-732d-4c7c-abad-1b2d853bc3c4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:44.398210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0e652-e669-4d5c-b4e1-6c99220b30ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:44.398805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e652-e669-4d5c-b4e1-6c99220b30ed admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.399080 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e652-e669-4d5c-b4e1-6c99220b30ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.399345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e652-e669-4d5c-b4e1-6c99220b30ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.400451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0e652-e669-4d5c-b4e1-6c99220b30ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:44.402562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0e652-e669-4d5c-b4e1-6c99220b30ed admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:44.476433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1065/2126] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:44 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:44.481114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30d679-df04-4bd4-8676-e9ca1061d70e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:44.492784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30d679-df04-4bd4-8676-e9ca1061d70e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:44.547701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e8a29-e043-4104-b458-300535dff3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:44.548254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e8a29-e043-4104-b458-300535dff3cd admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.548551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e8a29-e043-4104-b458-300535dff3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.548822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e8a29-e043-4104-b458-300535dff3cd admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.549454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e8a29-e043-4104-b458-300535dff3cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:44.551400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e8a29-e043-4104-b458-300535dff3cd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:44.629585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1062/2127] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:44 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:44.634572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0e652-e669-4d5c-b4e1-6c99220b30ed admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:44.646162 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0e652-e669-4d5c-b4e1-6c99220b30ed admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:44.651563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520ace-e33a-410f-8ab3-6206c472b39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:44.652152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520ace-e33a-410f-8ab3-6206c472b39f admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.652419 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520ace-e33a-410f-8ab3-6206c472b39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.652707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520ace-e33a-410f-8ab3-6206c472b39f admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.653334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520ace-e33a-410f-8ab3-6206c472b39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:44.654912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520ace-e33a-410f-8ab3-6206c472b39f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:44.665943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1066/2128] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:44.670611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e8a29-e043-4104-b458-300535dff3cd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:44.681435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e8a29-e043-4104-b458-300535dff3cd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:44.686256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d61aa-870d-465c-998b-35358ec3d195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:44.686905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d61aa-870d-465c-998b-35358ec3d195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.687182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d61aa-870d-465c-998b-35358ec3d195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.687453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d61aa-870d-465c-998b-35358ec3d195 admin admin] PATH_INFO: `/v3/projects/11d84a6665d440b98f25af37e7d994b9/users/aba60183e7de4b6e8febf4c5ac29854f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.704277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d61aa-870d-465c-998b-35358ec3d195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d84a6665d440b98f25af37e7d994b9, user_id=aba60183e7de4b6e8febf4c5ac29854f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:44.707009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d61aa-870d-465c-998b-35358ec3d195 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:44.766289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1063/2129] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:44 2022] PUT /identity/v3/projects/11d84a6665d440b98f25af37e7d994b9/users/aba60183e7de4b6e8febf4c5ac29854f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:44.771054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520ace-e33a-410f-8ab3-6206c472b39f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:44.782254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520ace-e33a-410f-8ab3-6206c472b39f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:44.846118 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5398af-7b47-4ca4-9ffa-1923e48dea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:44.846722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5398af-7b47-4ca4-9ffa-1923e48dea21 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.846996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5398af-7b47-4ca4-9ffa-1923e48dea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.847261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5398af-7b47-4ca4-9ffa-1923e48dea21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.847889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5398af-7b47-4ca4-9ffa-1923e48dea21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:44.849506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5398af-7b47-4ca4-9ffa-1923e48dea21 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:44.859249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1067/2130] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:44.863570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d61aa-870d-465c-998b-35358ec3d195 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:44.873770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d61aa-870d-465c-998b-35358ec3d195 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:44.878488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23107-1960-4926-b3af-7770fa6ac257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:44.879121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23107-1960-4926-b3af-7770fa6ac257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.879396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23107-1960-4926-b3af-7770fa6ac257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.879663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23107-1960-4926-b3af-7770fa6ac257 admin admin] PATH_INFO: `/v3/projects/11d84a6665d440b98f25af37e7d994b9/users/aba60183e7de4b6e8febf4c5ac29854f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.881774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a23107-1960-4926-b3af-7770fa6ac257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d84a6665d440b98f25af37e7d994b9, user_id=aba60183e7de4b6e8febf4c5ac29854f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:44.885754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a23107-1960-4926-b3af-7770fa6ac257 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:44.895171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1064/2131] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:44 2022] PUT /identity/v3/projects/11d84a6665d440b98f25af37e7d994b9/users/aba60183e7de4b6e8febf4c5ac29854f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:44.906359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe72f6-82ed-4446-8fca-eb18e124c8ce None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:44.913654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe72f6-82ed-4446-8fca-eb18e124c8ce None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:44.913654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe72f6-82ed-4446-8fca-eb18e124c8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:44.939348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e2fe72f6-82ed-4446-8fca-eb18e124c8ce None None] MFA Rules not processed for user `aba60183e7de4b6e8febf4c5ac29854f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:44.940564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e2fe72f6-82ed-4446-8fca-eb18e124c8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:44.959823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:45.019006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1068/2132] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:45.031111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23107-1960-4926-b3af-7770fa6ac257 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:45.032921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-45a23107-1960-4926-b3af-7770fa6ac257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:45.097072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23107-1960-4926-b3af-7770fa6ac257 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:45.100845 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4012f8-f8ea-43d8-b4b5-b235feeed536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:45.101438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4012f8-f8ea-43d8-b4b5-b235feeed536 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:45.101713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4012f8-f8ea-43d8-b4b5-b235feeed536 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:45.102029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4012f8-f8ea-43d8-b4b5-b235feeed536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:45.112314 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4012f8-f8ea-43d8-b4b5-b235feeed536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:45.124632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4012f8-f8ea-43d8-b4b5-b235feeed536 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:45.131397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1065/2133] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:45.831979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe72f6-82ed-4446-8fca-eb18e124c8ce None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:45.901721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe72f6-82ed-4446-8fca-eb18e124c8ce None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:45.906300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:45.906921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:45.907197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:45.907463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] PATH_INFO: `/v3/users/839dd931ca314062a1de67ca6b7d9bb6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:45.909213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] RBAC: Authorizing `identity:delete_user(user_id=839dd931ca314062a1de67ca6b7d9bb6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:45.910593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:46.081322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4012f8-f8ea-43d8-b4b5-b235feeed536 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:46.082774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5b4012f8-f8ea-43d8-b4b5-b235feeed536 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:46.097810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4012f8-f8ea-43d8-b4b5-b235feeed536 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:46.103043 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:46.103738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:46.104214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:46.104327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] PATH_INFO: `/v3/users/92119b2942df4136a5d3d436d2c54446` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:46.105420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] RBAC: Authorizing `identity:delete_user(user_id=92119b2942df4136a5d3d436d2c54446)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:46.107364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:46.112317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '839dd931ca314062a1de67ca6b7d9bb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.189517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '839dd931ca314062a1de67ca6b7d9bb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.198407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '839dd931ca314062a1de67ca6b7d9bb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.199360 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '839dd931ca314062a1de67ca6b7d9bb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.199999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ed1b6567-f477-4356-895b-f3884c9a371a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '839dd931ca314062a1de67ca6b7d9bb6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.218918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1069/2134] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:45 2022] DELETE /identity/v3/users/839dd931ca314062a1de67ca6b7d9bb6 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:46.223260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224f3e9-9d22-4463-a523-a3c3be5ac324 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:46.227939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224f3e9-9d22-4463-a523-a3c3be5ac324 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:46.227939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224f3e9-9d22-4463-a523-a3c3be5ac324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:46.244859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f224f3e9-9d22-4463-a523-a3c3be5ac324 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:46.246006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f224f3e9-9d22-4463-a523-a3c3be5ac324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:46.271444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:46.277573 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:46.309044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1070/2135] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:46.319298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224f3e9-9d22-4463-a523-a3c3be5ac324 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:46.320676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f224f3e9-9d22-4463-a523-a3c3be5ac324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:46.357951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92119b2942df4136a5d3d436d2c54446'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.377566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224f3e9-9d22-4463-a523-a3c3be5ac324 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:46.381307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71900b8a-e413-4a65-8101-43d828a65591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:46.381905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71900b8a-e413-4a65-8101-43d828a65591 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:46.382178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71900b8a-e413-4a65-8101-43d828a65591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:46.382452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71900b8a-e413-4a65-8101-43d828a65591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:46.392709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71900b8a-e413-4a65-8101-43d828a65591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:46.394187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71900b8a-e413-4a65-8101-43d828a65591 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:46.404216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1071/2136] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:46.440701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92119b2942df4136a5d3d436d2c54446'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.449181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92119b2942df4136a5d3d436d2c54446'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.449839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92119b2942df4136a5d3d436d2c54446'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.450499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92119b2942df4136a5d3d436d2c54446'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:46.486418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1066/2137] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:46 2022] DELETE /identity/v3/users/92119b2942df4136a5d3d436d2c54446 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:46.488122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201077a1-9b03-4028-9664-506a4e9eaef9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:46.488404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201077a1-9b03-4028-9664-506a4e9eaef9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:46.488697 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201077a1-9b03-4028-9664-506a4e9eaef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:46.510719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-201077a1-9b03-4028-9664-506a4e9eaef9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:46.520858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-201077a1-9b03-4028-9664-506a4e9eaef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:46.576798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1072/2138] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:46.582119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:46.583590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:46.667403 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe877-3225-47b7-8bcc-ec015b8abaea admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:46.671854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff6b0c-e29d-4d38-bfcf-8a55881a994a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:46.672382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff6b0c-e29d-4d38-bfcf-8a55881a994a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:46.672816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff6b0c-e29d-4d38-bfcf-8a55881a994a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:46.673088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff6b0c-e29d-4d38-bfcf-8a55881a994a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:46.684875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff6b0c-e29d-4d38-bfcf-8a55881a994a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:46.686438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff6b0c-e29d-4d38-bfcf-8a55881a994a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:46.697628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1067/2139] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:46.934551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201077a1-9b03-4028-9664-506a4e9eaef9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:46.936050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-201077a1-9b03-4028-9664-506a4e9eaef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:46.948680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201077a1-9b03-4028-9664-506a4e9eaef9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:46.954259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:46.955083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:46.955349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:46.955616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] PATH_INFO: `/v3/projects/c976e4ae291343b6a1959563d936df2d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:46.957107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c976e4ae291343b6a1959563d936df2d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:46.961478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:47.098119 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c976e4ae291343b6a1959563d936df2d', 'request_id': 'req-1ae9f19d-603b-474d-a73b-c523b37e2614'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.116781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c976e4ae291343b6a1959563d936df2d', 'request_id': 'req-1ae9f19d-603b-474d-a73b-c523b37e2614'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.240629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c976e4ae291343b6a1959563d936df2d', 'request_id': 'req-1ae9f19d-603b-474d-a73b-c523b37e2614'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.253068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] The token cache is being invalidate because project c976e4ae291343b6a1959563d936df2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:47.253353 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.253980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.256969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1073/2140] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:46 2022] DELETE /identity/v3/projects/c976e4ae291343b6a1959563d936df2d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:47.260463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff6b0c-e29d-4d38-bfcf-8a55881a994a service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:47.271062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff6b0c-e29d-4d38-bfcf-8a55881a994a service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:47.299852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:47.305113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:47.307509 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:47.341128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:47.341776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:47.342022 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:47.342281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] PATH_INFO: `/v3/users/aba60183e7de4b6e8febf4c5ac29854f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:47.343432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba60183e7de4b6e8febf4c5ac29854f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:47.345438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:47.372028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9f19d-603b-474d-a73b-c523b37e2614 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:47.377536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:47.378226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:47.378506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:47.378777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] PATH_INFO: `/v3/projects/d4d9af0e1c65440f816a850317372e1b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:47.379991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d9af0e1c65440f816a850317372e1b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:47.381992 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:47.586830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba60183e7de4b6e8febf4c5ac29854f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.587526 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d9af0e1c65440f816a850317372e1b', 'request_id': 'req-a778b285-7026-451b-abd9-c64ae0f66c6e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.593158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:47.593900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d9af0e1c65440f816a850317372e1b', 'request_id': 'req-a778b285-7026-451b-abd9-c64ae0f66c6e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.686548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba60183e7de4b6e8febf4c5ac29854f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.695122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba60183e7de4b6e8febf4c5ac29854f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.695805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba60183e7de4b6e8febf4c5ac29854f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.697154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47022496-2702-4bac-b388-88704f3c64aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba60183e7de4b6e8febf4c5ac29854f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.722636 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1068/2141] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:10:47 2022] DELETE /identity/v3/users/aba60183e7de4b6e8febf4c5ac29854f => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:47.766570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d9af0e1c65440f816a850317372e1b', 'request_id': 'req-a778b285-7026-451b-abd9-c64ae0f66c6e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.781478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] The token cache is being invalidate because project d4d9af0e1c65440f816a850317372e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:47.781787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.782476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a778b285-7026-451b-abd9-c64ae0f66c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:47.787841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1074/2142] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:47 2022] DELETE /identity/v3/projects/d4d9af0e1c65440f816a850317372e1b => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:47.789318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1689c3-d238-4878-8d05-7e43fb78845b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:47.789603 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1689c3-d238-4878-8d05-7e43fb78845b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:47.789872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1689c3-d238-4878-8d05-7e43fb78845b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:47.815335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-bc1689c3-d238-4878-8d05-7e43fb78845b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:47.816705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bc1689c3-d238-4878-8d05-7e43fb78845b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:47.893135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1069/2143] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:47.901916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe077-8dd2-45b8-96c3-f75942d34b45 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:47.902198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe077-8dd2-45b8-96c3-f75942d34b45 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:47.902468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe077-8dd2-45b8-96c3-f75942d34b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:47.922304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c0cfe077-8dd2-45b8-96c3-f75942d34b45 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:47.923506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c0cfe077-8dd2-45b8-96c3-f75942d34b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:47.931891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1075/2144] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:47.935942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1689c3-d238-4878-8d05-7e43fb78845b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:47.951456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1689c3-d238-4878-8d05-7e43fb78845b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:47.956950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44b127-00f7-491b-a8c3-3c3e16683b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:47.957574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44b127-00f7-491b-a8c3-3c3e16683b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:47.957850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44b127-00f7-491b-a8c3-3c3e16683b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:47.958114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44b127-00f7-491b-a8c3-3c3e16683b6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:47.958778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44b127-00f7-491b-a8c3-3c3e16683b6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:47.959068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44b127-00f7-491b-a8c3-3c3e16683b6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:47.960723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44b127-00f7-491b-a8c3-3c3e16683b6f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:47.991387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1070/2145] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:10:47.998159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfe077-8dd2-45b8-96c3-f75942d34b45 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.007634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfe077-8dd2-45b8-96c3-f75942d34b45 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.012596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c872f-946e-4aeb-8e34-1931c61b93f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.013190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c872f-946e-4aeb-8e34-1931c61b93f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.013463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c872f-946e-4aeb-8e34-1931c61b93f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.013730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c872f-946e-4aeb-8e34-1931c61b93f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.014377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c872f-946e-4aeb-8e34-1931c61b93f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.016278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c872f-946e-4aeb-8e34-1931c61b93f8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:48.114490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1076/2146] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:47 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:48.116169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44b127-00f7-491b-a8c3-3c3e16683b6f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.126532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44b127-00f7-491b-a8c3-3c3e16683b6f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.202655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5ea19-22cf-4100-a32f-a449fcccaa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.203654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5ea19-22cf-4100-a32f-a449fcccaa19 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.203960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5ea19-22cf-4100-a32f-a449fcccaa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.204203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5ea19-22cf-4100-a32f-a449fcccaa19 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.204909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5ea19-22cf-4100-a32f-a449fcccaa19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.206832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5ea19-22cf-4100-a32f-a449fcccaa19 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:48.255225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c872f-946e-4aeb-8e34-1931c61b93f8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.257249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-594c872f-946e-4aeb-8e34-1931c61b93f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:48.273790 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c872f-946e-4aeb-8e34-1931c61b93f8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.280226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.285471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.285471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.285471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] PATH_INFO: `/v3/projects/11d84a6665d440b98f25af37e7d994b9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.285471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d84a6665d440b98f25af37e7d994b9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.286424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:48.332422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1071/2147] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:48 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:48.336660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5ea19-22cf-4100-a32f-a449fcccaa19 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.347310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5ea19-22cf-4100-a32f-a449fcccaa19 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.352185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811356e4-cc3e-4bbf-b7bb-b8a70a495cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.353392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811356e4-cc3e-4bbf-b7bb-b8a70a495cba admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.353673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811356e4-cc3e-4bbf-b7bb-b8a70a495cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.353943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811356e4-cc3e-4bbf-b7bb-b8a70a495cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.354588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811356e4-cc3e-4bbf-b7bb-b8a70a495cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.356197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811356e4-cc3e-4bbf-b7bb-b8a70a495cba admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:48.367389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1072/2148] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:10:48.373143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811356e4-cc3e-4bbf-b7bb-b8a70a495cba admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.383943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811356e4-cc3e-4bbf-b7bb-b8a70a495cba admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.389436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c1ae1-797b-4cd6-9d6a-6e41a15519ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.389436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c1ae1-797b-4cd6-9d6a-6e41a15519ac admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.389770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c1ae1-797b-4cd6-9d6a-6e41a15519ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.390036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c1ae1-797b-4cd6-9d6a-6e41a15519ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.390665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c1ae1-797b-4cd6-9d6a-6e41a15519ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.392229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c1ae1-797b-4cd6-9d6a-6e41a15519ac admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:48.403136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1073/2149] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:48.407600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c1ae1-797b-4cd6-9d6a-6e41a15519ac admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.422210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c1ae1-797b-4cd6-9d6a-6e41a15519ac admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.436649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d84a6665d440b98f25af37e7d994b9', 'request_id': 'req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:48.442555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d84a6665d440b98f25af37e7d994b9', 'request_id': 'req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:48.451226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:48.483044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e58df-06ff-4689-a69a-37a33e5fde34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.483756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e58df-06ff-4689-a69a-37a33e5fde34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.486346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e58df-06ff-4689-a69a-37a33e5fde34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.486346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e58df-06ff-4689-a69a-37a33e5fde34 admin admin] PATH_INFO: `/v3/projects/c3b1914ba55849e4a3a55a20ed50e7a1/users/6e4ebbf55d214dcc835e3f43876693bb/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.499708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e58df-06ff-4689-a69a-37a33e5fde34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b1914ba55849e4a3a55a20ed50e7a1, user_id=6e4ebbf55d214dcc835e3f43876693bb, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.502610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e58df-06ff-4689-a69a-37a33e5fde34 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:48.506282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:48.538066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d84a6665d440b98f25af37e7d994b9', 'request_id': 'req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:48.550120 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] The token cache is being invalidate because project 11d84a6665d440b98f25af37e7d994b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:10:48.550547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:48.551211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-3192cccf-0c21-4d3f-baa9-678a60fc7e3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:10:48.552749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1077/2150] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:10:48 2022] DELETE /identity/v3/projects/11d84a6665d440b98f25af37e7d994b9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:48.557423 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99f6df-b902-4e5f-85fa-1ee73aa15872 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.557809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99f6df-b902-4e5f-85fa-1ee73aa15872 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.557972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99f6df-b902-4e5f-85fa-1ee73aa15872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.581294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6e99f6df-b902-4e5f-85fa-1ee73aa15872 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:48.582490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6e99f6df-b902-4e5f-85fa-1ee73aa15872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:48.591064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1078/2151] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:48.598679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a301968-80f2-4bc3-8c2e-705fd1dd2137 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.598957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a301968-80f2-4bc3-8c2e-705fd1dd2137 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.599243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a301968-80f2-4bc3-8c2e-705fd1dd2137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.618079 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4a301968-80f2-4bc3-8c2e-705fd1dd2137 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:48.619234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4a301968-80f2-4bc3-8c2e-705fd1dd2137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:48.629416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1074/2152] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:48 2022] PUT /identity/v3/projects/c3b1914ba55849e4a3a55a20ed50e7a1/users/6e4ebbf55d214dcc835e3f43876693bb/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:10:48.634917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204f7fd-90d2-4915-8a00-9e9bfe24b11b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.635188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204f7fd-90d2-4915-8a00-9e9bfe24b11b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.635468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204f7fd-90d2-4915-8a00-9e9bfe24b11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.655660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9204f7fd-90d2-4915-8a00-9e9bfe24b11b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:48.656892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9204f7fd-90d2-4915-8a00-9e9bfe24b11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:48.684898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1079/2153] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:10:48.688969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a301968-80f2-4bc3-8c2e-705fd1dd2137 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.698677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a301968-80f2-4bc3-8c2e-705fd1dd2137 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.704798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.705531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.705850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.706068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.706829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:10:48.707175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.708794 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:48.718694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1075/2154] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:48.726159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1080/2155] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:10:48.727220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204f7fd-90d2-4915-8a00-9e9bfe24b11b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.733155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.734910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:48.740375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204f7fd-90d2-4915-8a00-9e9bfe24b11b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.745467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e697c-38a1-48d0-939e-51491253c36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.746433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e697c-38a1-48d0-939e-51491253c36b admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.746558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e697c-38a1-48d0-939e-51491253c36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.747697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e697c-38a1-48d0-939e-51491253c36b admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.747697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e697c-38a1-48d0-939e-51491253c36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.749560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e697c-38a1-48d0-939e-51491253c36b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:48.795815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbb09c-9d1b-4e6d-adda-8149af04868e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.799730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3c904-6bd1-4ae3-b0cf-8d8d5a612eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.800323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3c904-6bd1-4ae3-b0cf-8d8d5a612eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.800627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3c904-6bd1-4ae3-b0cf-8d8d5a612eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.800891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3c904-6bd1-4ae3-b0cf-8d8d5a612eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.810140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3c904-6bd1-4ae3-b0cf-8d8d5a612eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.811580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3c904-6bd1-4ae3-b0cf-8d8d5a612eb5 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:48.821180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1081/2156] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:10:48.899073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1076/2157] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:48 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:10:48.903707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3c904-6bd1-4ae3-b0cf-8d8d5a612eb5 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:48.913376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3c904-6bd1-4ae3-b0cf-8d8d5a612eb5 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:48.973442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37a235-8d4b-4f08-8fa3-d758048b4519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:48.974048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37a235-8d4b-4f08-8fa3-d758048b4519 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:48.974326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37a235-8d4b-4f08-8fa3-d758048b4519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:48.974592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37a235-8d4b-4f08-8fa3-d758048b4519 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:48.975346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37a235-8d4b-4f08-8fa3-d758048b4519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:48.977305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37a235-8d4b-4f08-8fa3-d758048b4519 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:49.088393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1082/2158] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:48 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:49.090003 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e697c-38a1-48d0-939e-51491253c36b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:49.102790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e697c-38a1-48d0-939e-51491253c36b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:49.108265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d50b73-4438-41b1-b893-afba2aa6e5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:49.108891 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d50b73-4438-41b1-b893-afba2aa6e5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:49.109185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d50b73-4438-41b1-b893-afba2aa6e5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:49.109447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d50b73-4438-41b1-b893-afba2aa6e5f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:49.110075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d50b73-4438-41b1-b893-afba2aa6e5f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:49.111648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d50b73-4438-41b1-b893-afba2aa6e5f9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:49.121491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1077/2159] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:49.125564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37a235-8d4b-4f08-8fa3-d758048b4519 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:49.136305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37a235-8d4b-4f08-8fa3-d758048b4519 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:49.141066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3f810-b7df-4703-8d52-c4b4fe03b70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:49.141775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f810-b7df-4703-8d52-c4b4fe03b70e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:49.142026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f810-b7df-4703-8d52-c4b4fe03b70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:49.142319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f810-b7df-4703-8d52-c4b4fe03b70e admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:49.142956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3f810-b7df-4703-8d52-c4b4fe03b70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:49.144597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3f810-b7df-4703-8d52-c4b4fe03b70e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:49.155078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1083/2160] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:49.159059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d50b73-4438-41b1-b893-afba2aa6e5f9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:49.169087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d50b73-4438-41b1-b893-afba2aa6e5f9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:49.173864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02253e1-cb90-498f-a85c-2f1386298abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:49.174508 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02253e1-cb90-498f-a85c-2f1386298abc admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:49.174791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02253e1-cb90-498f-a85c-2f1386298abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:49.175055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02253e1-cb90-498f-a85c-2f1386298abc admin admin] PATH_INFO: `/v3/projects/2cedb1ef78764683bdb6888501b18733/users/1d0af4ed24d0400ea94fa3edadac264b/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:49.191065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02253e1-cb90-498f-a85c-2f1386298abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cedb1ef78764683bdb6888501b18733, user_id=1d0af4ed24d0400ea94fa3edadac264b, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:49.194425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02253e1-cb90-498f-a85c-2f1386298abc admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:49.282053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1078/2161] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:49 2022] PUT /identity/v3/projects/2cedb1ef78764683bdb6888501b18733/users/1d0af4ed24d0400ea94fa3edadac264b/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:49.295978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6b0e8-ebbd-4254-9a16-614806e52144 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:49.296287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6b0e8-ebbd-4254-9a16-614806e52144 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:49.296784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6b0e8-ebbd-4254-9a16-614806e52144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:49.325018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-09e6b0e8-ebbd-4254-9a16-614806e52144 None None] MFA Rules not processed for user `1d0af4ed24d0400ea94fa3edadac264b`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:49.326218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-09e6b0e8-ebbd-4254-9a16-614806e52144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:49.503866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1084/2162] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:49.505500 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02253e1-cb90-498f-a85c-2f1386298abc admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:49.506696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f02253e1-cb90-498f-a85c-2f1386298abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:49.567239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:49.591986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02253e1-cb90-498f-a85c-2f1386298abc admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:49.596389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e174cd-8ec1-4490-9471-578095f777a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:49.597295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e174cd-8ec1-4490-9471-578095f777a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:49.597743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e174cd-8ec1-4490-9471-578095f777a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:49.598013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e174cd-8ec1-4490-9471-578095f777a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:49.609507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e174cd-8ec1-4490-9471-578095f777a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:49.611077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e174cd-8ec1-4490-9471-578095f777a3 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:49.620015 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1079/2163] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:10:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:10:49.623837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:54.828025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797ca3b-528a-44ae-912b-0fbd1e254e0b None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:54.828594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797ca3b-528a-44ae-912b-0fbd1e254e0b None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:54.828594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797ca3b-528a-44ae-912b-0fbd1e254e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:54.872617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8797ca3b-528a-44ae-912b-0fbd1e254e0b None None] MFA Rules not processed for user `6e4ebbf55d214dcc835e3f43876693bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:10:54.873871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8797ca3b-528a-44ae-912b-0fbd1e254e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:54.943364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1085/2164] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:10:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:10:54.949761 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e174cd-8ec1-4490-9471-578095f777a3 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:54.951334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-97e174cd-8ec1-4490-9471-578095f777a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:10:55.035244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e174cd-8ec1-4490-9471-578095f777a3 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:55.037466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964c910-1524-4d6f-9f44-34a9ed77806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:55.038156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964c910-1524-4d6f-9f44-34a9ed77806a admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:55.038436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964c910-1524-4d6f-9f44-34a9ed77806a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:55.038732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964c910-1524-4d6f-9f44-34a9ed77806a admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:55.039347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964c910-1524-4d6f-9f44-34a9ed77806a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:55.043337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964c910-1524-4d6f-9f44-34a9ed77806a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:55.158388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1080/2165] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:10:54 2022] POST /identity/v3/projects => generated 383 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:55.162420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797ca3b-528a-44ae-912b-0fbd1e254e0b None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:55.173687 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797ca3b-528a-44ae-912b-0fbd1e254e0b None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:55.242702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392629c1-337e-46cc-9b5a-be1b07878ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:55.243338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392629c1-337e-46cc-9b5a-be1b07878ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:55.243653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392629c1-337e-46cc-9b5a-be1b07878ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:55.243933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392629c1-337e-46cc-9b5a-be1b07878ff1 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:55.244646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392629c1-337e-46cc-9b5a-be1b07878ff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:55.246528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392629c1-337e-46cc-9b5a-be1b07878ff1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:55.309646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:10:55.315961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1086/2166] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:10:55 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:10:55.320137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964c910-1524-4d6f-9f44-34a9ed77806a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:55.332924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964c910-1524-4d6f-9f44-34a9ed77806a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:55.338215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657bf7da-03f3-4718-8dcd-8b3190a42d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:55.338805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bf7da-03f3-4718-8dcd-8b3190a42d21 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:55.339079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bf7da-03f3-4718-8dcd-8b3190a42d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:55.339343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bf7da-03f3-4718-8dcd-8b3190a42d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:55.339975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bf7da-03f3-4718-8dcd-8b3190a42d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:55.341594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bf7da-03f3-4718-8dcd-8b3190a42d21 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:55.352207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1081/2167] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:55.358107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392629c1-337e-46cc-9b5a-be1b07878ff1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:55.368119 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392629c1-337e-46cc-9b5a-be1b07878ff1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:55.373538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064eae7-1688-48a0-91e1-dee272b4762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:55.374189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064eae7-1688-48a0-91e1-dee272b4762f admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:55.374466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064eae7-1688-48a0-91e1-dee272b4762f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:55.374737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064eae7-1688-48a0-91e1-dee272b4762f admin admin] PATH_INFO: `/v3/projects/df7079a696d54eb7bc8e9d4830077b37/users/5094cabe5ad9475da8a3e2b6cd808117/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:55.389980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064eae7-1688-48a0-91e1-dee272b4762f admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7079a696d54eb7bc8e9d4830077b37, user_id=5094cabe5ad9475da8a3e2b6cd808117, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:55.392741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064eae7-1688-48a0-91e1-dee272b4762f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:55.466383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1087/2168] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:55 2022] PUT /identity/v3/projects/df7079a696d54eb7bc8e9d4830077b37/users/5094cabe5ad9475da8a3e2b6cd808117/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:10:55.471814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657bf7da-03f3-4718-8dcd-8b3190a42d21 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:55.483442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657bf7da-03f3-4718-8dcd-8b3190a42d21 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:55.538221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb0861-2712-404a-9955-edec78de9242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:55.538814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb0861-2712-404a-9955-edec78de9242 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:55.539086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb0861-2712-404a-9955-edec78de9242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:55.539353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb0861-2712-404a-9955-edec78de9242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:55.539983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb0861-2712-404a-9955-edec78de9242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:55.541591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb0861-2712-404a-9955-edec78de9242 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:55.551064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1082/2169] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:10:55.555655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064eae7-1688-48a0-91e1-dee272b4762f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:10:55.565775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064eae7-1688-48a0-91e1-dee272b4762f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:10:55.570232 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d069772-8ec1-4736-929d-2b7b1ec7f9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:10:55.571535 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069772-8ec1-4736-929d-2b7b1ec7f9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:10:55.571909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069772-8ec1-4736-929d-2b7b1ec7f9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:10:55.572184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069772-8ec1-4736-929d-2b7b1ec7f9ae admin admin] PATH_INFO: `/v3/projects/df7079a696d54eb7bc8e9d4830077b37/users/5094cabe5ad9475da8a3e2b6cd808117/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:10:55.574376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d069772-8ec1-4736-929d-2b7b1ec7f9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7079a696d54eb7bc8e9d4830077b37, user_id=5094cabe5ad9475da8a3e2b6cd808117, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:10:55.577028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d069772-8ec1-4736-929d-2b7b1ec7f9ae admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:10:55.586888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1088/2170] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:10:55 2022] PUT /identity/v3/projects/df7079a696d54eb7bc8e9d4830077b37/users/5094cabe5ad9475da8a3e2b6cd808117/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:11:01.031614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03b061-8ea4-4a74-9d9a-67894654af2f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:01.032071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03b061-8ea4-4a74-9d9a-67894654af2f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:01.032346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03b061-8ea4-4a74-9d9a-67894654af2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:01.059850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ec03b061-8ea4-4a74-9d9a-67894654af2f None None] MFA Rules not processed for user `2d5e7d5c77084705b1f39406293ab585`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:01.061283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ec03b061-8ea4-4a74-9d9a-67894654af2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:01.127915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1083/2171] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:03.376775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0f3f4-cef2-467d-ac7a-c7c77033bc12 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:03.377261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0f3f4-cef2-467d-ac7a-c7c77033bc12 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:03.377342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0f3f4-cef2-467d-ac7a-c7c77033bc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:03.411034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-84e0f3f4-cef2-467d-ac7a-c7c77033bc12 None None] MFA Rules not processed for user `5094cabe5ad9475da8a3e2b6cd808117`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:03.412340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-84e0f3f4-cef2-467d-ac7a-c7c77033bc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:03.491271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1089/2172] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:03.506853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03b061-8ea4-4a74-9d9a-67894654af2f None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:03.509128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ec03b061-8ea4-4a74-9d9a-67894654af2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:03.519117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:03.566558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03b061-8ea4-4a74-9d9a-67894654af2f None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:03.571045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3c49f-6da1-40ca-9ccf-a7fa84ce580f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e95d295337b4472be946d252ac3bd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:03.571651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3c49f-6da1-40ca-9ccf-a7fa84ce580f service glance] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:03.577411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3c49f-6da1-40ca-9ccf-a7fa84ce580f service glance] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:03.577411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3c49f-6da1-40ca-9ccf-a7fa84ce580f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:03.634110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3c49f-6da1-40ca-9ccf-a7fa84ce580f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:03.635575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3c49f-6da1-40ca-9ccf-a7fa84ce580f service glance] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:03.645059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1084/2173] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:11:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:11:03.946544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c01934-ac44-4bdb-9bd1-59e9913ad173 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:03.946824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c01934-ac44-4bdb-9bd1-59e9913ad173 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:03.947089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c01934-ac44-4bdb-9bd1-59e9913ad173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:03.969115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-33c01934-ac44-4bdb-9bd1-59e9913ad173 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:03.969115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-33c01934-ac44-4bdb-9bd1-59e9913ad173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:04.021516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1090/2174] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:11:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:04.062757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3c49f-6da1-40ca-9ccf-a7fa84ce580f service glance] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:04.064181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4db3c49f-6da1-40ca-9ccf-a7fa84ce580f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:04.133698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3c49f-6da1-40ca-9ccf-a7fa84ce580f service glance] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:04.138915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ef2e2-db97-4a35-9f16-4eb394d26b23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:04.139524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ef2e2-db97-4a35-9f16-4eb394d26b23 service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:04.139792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ef2e2-db97-4a35-9f16-4eb394d26b23 service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:04.140059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ef2e2-db97-4a35-9f16-4eb394d26b23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:04.149824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ef2e2-db97-4a35-9f16-4eb394d26b23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:04.151443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ef2e2-db97-4a35-9f16-4eb394d26b23 service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:04.161188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1085/2175] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:11:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:11:11.304457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c01934-ac44-4bdb-9bd1-59e9913ad173 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:11.310825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-33c01934-ac44-4bdb-9bd1-59e9913ad173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:11.411335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:11.429961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c01934-ac44-4bdb-9bd1-59e9913ad173 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:11.435440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:11.436298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:11.436747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:11.437246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] PATH_INFO: `/v3/users/3bb8dfebe63944f395d17ba532b807a3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:11.438556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb8dfebe63944f395d17ba532b807a3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:11.440673 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:11.645034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb8dfebe63944f395d17ba532b807a3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:11.697511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb8dfebe63944f395d17ba532b807a3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:11.718992 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb8dfebe63944f395d17ba532b807a3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:11.720458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb8dfebe63944f395d17ba532b807a3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:11.721393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb8dfebe63944f395d17ba532b807a3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:11.749802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1091/2176] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:11:11 2022] DELETE /identity/v3/users/3bb8dfebe63944f395d17ba532b807a3 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:11:11.754880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fce0-cf1f-4231-a256-1688b0f9dd69 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:11.755443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fce0-cf1f-4231-a256-1688b0f9dd69 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:11.755443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fce0-cf1f-4231-a256-1688b0f9dd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:11.777966 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0319fce0-cf1f-4231-a256-1688b0f9dd69 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:11.779198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0319fce0-cf1f-4231-a256-1688b0f9dd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:11.850406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:11.870755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1086/2177] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:11.884590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:11.886171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:11.970037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8c419-d9cf-403c-843a-5d7ac41a7823 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:11.974662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceb800-eba5-4238-8240-811dfdd18fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:11.975343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceb800-eba5-4238-8240-811dfdd18fbd service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:11.975679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceb800-eba5-4238-8240-811dfdd18fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:11.976025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceb800-eba5-4238-8240-811dfdd18fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:11.988753 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceb800-eba5-4238-8240-811dfdd18fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:11.990966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceb800-eba5-4238-8240-811dfdd18fbd service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:12.003379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1092/2178] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:11:12.551641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fce0-cf1f-4231-a256-1688b0f9dd69 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:12.553545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0319fce0-cf1f-4231-a256-1688b0f9dd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:12.577016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fce0-cf1f-4231-a256-1688b0f9dd69 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:12.597283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:12.599458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:12.599919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:12.601187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] PATH_INFO: `/v3/projects/72d20e1627fa4f72857d077c2bc15c33` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:12.602605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d20e1627fa4f72857d077c2bc15c33)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:12.605810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:12.760657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d20e1627fa4f72857d077c2bc15c33', 'request_id': 'req-7536689e-eb49-4abb-9d4c-a7e847469e34'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:12.767731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d20e1627fa4f72857d077c2bc15c33', 'request_id': 'req-7536689e-eb49-4abb-9d4c-a7e847469e34'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:12.834335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d20e1627fa4f72857d077c2bc15c33', 'request_id': 'req-7536689e-eb49-4abb-9d4c-a7e847469e34'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:12.853683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] The token cache is being invalidate because project 72d20e1627fa4f72857d077c2bc15c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:11:12.854026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:12.854638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7536689e-eb49-4abb-9d4c-a7e847469e34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:12.856548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1087/2179] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:11:12 2022] DELETE /identity/v3/projects/72d20e1627fa4f72857d077c2bc15c33 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:11:12.861956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba1a9e-7297-4388-9651-b7e672f099e0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:12.862506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba1a9e-7297-4388-9651-b7e672f099e0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:12.862616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba1a9e-7297-4388-9651-b7e672f099e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:12.888148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-72ba1a9e-7297-4388-9651-b7e672f099e0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:12.889508 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-72ba1a9e-7297-4388-9651-b7e672f099e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:12.967701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1093/2180] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:12.975900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d043e63-abb4-4cc8-b37f-dc7fa1ff357a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:12.979545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d043e63-abb4-4cc8-b37f-dc7fa1ff357a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:12.979545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d043e63-abb4-4cc8-b37f-dc7fa1ff357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:13.002055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3d043e63-abb4-4cc8-b37f-dc7fa1ff357a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:13.003289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3d043e63-abb4-4cc8-b37f-dc7fa1ff357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:13.012240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1088/2181] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:13.016722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba1a9e-7297-4388-9651-b7e672f099e0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:13.031607 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba1a9e-7297-4388-9651-b7e672f099e0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:13.040300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b740cba-9d8a-4433-9dc7-45da11601cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:13.040967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b740cba-9d8a-4433-9dc7-45da11601cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:13.041258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b740cba-9d8a-4433-9dc7-45da11601cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:13.044600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b740cba-9d8a-4433-9dc7-45da11601cc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:13.044600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b740cba-9d8a-4433-9dc7-45da11601cc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:11:13.044600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b740cba-9d8a-4433-9dc7-45da11601cc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:13.044600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b740cba-9d8a-4433-9dc7-45da11601cc6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:13.054385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:13.058351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1094/2182] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:11:13.062867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d043e63-abb4-4cc8-b37f-dc7fa1ff357a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:13.072728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d043e63-abb4-4cc8-b37f-dc7fa1ff357a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:13.078738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114aa46-fc79-4b8a-819a-612aa4a0d1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:13.079335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114aa46-fc79-4b8a-819a-612aa4a0d1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:13.079608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114aa46-fc79-4b8a-819a-612aa4a0d1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:13.079874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114aa46-fc79-4b8a-819a-612aa4a0d1e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:13.080567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114aa46-fc79-4b8a-819a-612aa4a0d1e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:13.082480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114aa46-fc79-4b8a-819a-612aa4a0d1e5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:13.717370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1089/2183] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:11:13 2022] POST /identity/v3/projects => generated 373 bytes in 652 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:11:13.718769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b740cba-9d8a-4433-9dc7-45da11601cc6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:13.729857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b740cba-9d8a-4433-9dc7-45da11601cc6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:13.821878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0acdcc8-b954-40b1-8662-bf7131db64df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:13.822483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0acdcc8-b954-40b1-8662-bf7131db64df admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:13.822760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0acdcc8-b954-40b1-8662-bf7131db64df admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:13.823047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0acdcc8-b954-40b1-8662-bf7131db64df admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:13.823977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0acdcc8-b954-40b1-8662-bf7131db64df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:13.826003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0acdcc8-b954-40b1-8662-bf7131db64df admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:14.038137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1095/2184] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:11:13 2022] POST /identity/v3/users => generated 287 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:11:14.039454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114aa46-fc79-4b8a-819a-612aa4a0d1e5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:14.051681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114aa46-fc79-4b8a-819a-612aa4a0d1e5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:14.057581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00892-c04d-451b-9848-51a6d6acd704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:14.058290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00892-c04d-451b-9848-51a6d6acd704 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:14.058507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00892-c04d-451b-9848-51a6d6acd704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:14.059473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00892-c04d-451b-9848-51a6d6acd704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:14.060020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb00892-c04d-451b-9848-51a6d6acd704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:14.061715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb00892-c04d-451b-9848-51a6d6acd704 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:14.077445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1090/2185] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:11:14.081383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0acdcc8-b954-40b1-8662-bf7131db64df admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:14.093952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0acdcc8-b954-40b1-8662-bf7131db64df admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:14.099296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803df191-ff01-4995-a226-6f2253844e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:14.100041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803df191-ff01-4995-a226-6f2253844e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:14.104586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803df191-ff01-4995-a226-6f2253844e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:14.104586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803df191-ff01-4995-a226-6f2253844e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:14.104586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803df191-ff01-4995-a226-6f2253844e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:14.104586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803df191-ff01-4995-a226-6f2253844e0d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:14.128894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1096/2186] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:11:14.133446 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00892-c04d-451b-9848-51a6d6acd704 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:14.145879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00892-c04d-451b-9848-51a6d6acd704 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:14.151554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4939fbc-fbc5-467a-8ce3-517e017043be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:14.152197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4939fbc-fbc5-467a-8ce3-517e017043be admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:14.152476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4939fbc-fbc5-467a-8ce3-517e017043be admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:14.153793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4939fbc-fbc5-467a-8ce3-517e017043be admin admin] PATH_INFO: `/v3/projects/88923ed488964be98362489d41fe1875/users/ada859661bef4598b4e8d039b70efcdd/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:14.170832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4939fbc-fbc5-467a-8ce3-517e017043be admin admin] RBAC: Authorizing `identity:create_grant(project_id=88923ed488964be98362489d41fe1875, user_id=ada859661bef4598b4e8d039b70efcdd, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:14.173848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4939fbc-fbc5-467a-8ce3-517e017043be admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:14.231530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1091/2187] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:11:14 2022] PUT /identity/v3/projects/88923ed488964be98362489d41fe1875/users/ada859661bef4598b4e8d039b70efcdd/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:11:14.242782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a75ef-f3d3-4f77-969c-174574be2125 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:14.243068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a75ef-f3d3-4f77-969c-174574be2125 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:14.243341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a75ef-f3d3-4f77-969c-174574be2125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:14.283241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-088a75ef-f3d3-4f77-969c-174574be2125 None None] MFA Rules not processed for user `ada859661bef4598b4e8d039b70efcdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:14.285216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-088a75ef-f3d3-4f77-969c-174574be2125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:14.370549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1097/2188] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:14.385496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4939fbc-fbc5-467a-8ce3-517e017043be admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:14.394031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e4939fbc-fbc5-467a-8ce3-517e017043be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:14.477624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4939fbc-fbc5-467a-8ce3-517e017043be admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:14.483550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7ffae-e45f-48d9-ae4d-ae2dc8ebc5bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:14.484433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7ffae-e45f-48d9-ae4d-ae2dc8ebc5bd service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:14.485056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7ffae-e45f-48d9-ae4d-ae2dc8ebc5bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:14.485432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7ffae-e45f-48d9-ae4d-ae2dc8ebc5bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:14.495695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7ffae-e45f-48d9-ae4d-ae2dc8ebc5bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:14.497512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7ffae-e45f-48d9-ae4d-ae2dc8ebc5bd service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:14.509137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1092/2189] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:11:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:11:14.564827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a75ef-f3d3-4f77-969c-174574be2125 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:14.675746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a75ef-f3d3-4f77-969c-174574be2125 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:14.683739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:14.684411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:14.684777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:14.690107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] PATH_INFO: `/v3/users/ada859661bef4598b4e8d039b70efcdd` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:14.690107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ada859661bef4598b4e8d039b70efcdd)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:14.690107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:14.700577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:14.937870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ada859661bef4598b4e8d039b70efcdd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:15.040726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ada859661bef4598b4e8d039b70efcdd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:15.050364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ada859661bef4598b4e8d039b70efcdd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:15.052579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ada859661bef4598b4e8d039b70efcdd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:15.053678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ada859661bef4598b4e8d039b70efcdd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:15.097771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1098/2190] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:11:14 2022] DELETE /identity/v3/users/ada859661bef4598b4e8d039b70efcdd => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:11:15.099054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ff0d0-6122-444e-8d6e-87a047a5e912 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:15.099525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ff0d0-6122-444e-8d6e-87a047a5e912 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:15.099627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ff0d0-6122-444e-8d6e-87a047a5e912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:15.144559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7c9ff0d0-6122-444e-8d6e-87a047a5e912 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:15.145882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7c9ff0d0-6122-444e-8d6e-87a047a5e912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:15.159468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:15.286692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1093/2191] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:11:15.312959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:15.314663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:15.439202 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b802aa-6297-4ec5-802a-2181b70fe633 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:15.448932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1f866-3c61-4502-aed7-2391357c0996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:15.450047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1f866-3c61-4502-aed7-2391357c0996 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:15.450560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1f866-3c61-4502-aed7-2391357c0996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:15.451109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1f866-3c61-4502-aed7-2391357c0996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:15.477640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1f866-3c61-4502-aed7-2391357c0996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:15.480034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1f866-3c61-4502-aed7-2391357c0996 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:15.500289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1099/2192] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:11:16.400983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ff0d0-6122-444e-8d6e-87a047a5e912 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:16.402564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7c9ff0d0-6122-444e-8d6e-87a047a5e912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:16.417311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ff0d0-6122-444e-8d6e-87a047a5e912 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:16.423333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:16.424370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:16.424892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:16.425362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] PATH_INFO: `/v3/projects/88923ed488964be98362489d41fe1875` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:16.431272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88923ed488964be98362489d41fe1875)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:16.434869 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:16.619529 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88923ed488964be98362489d41fe1875', 'request_id': 'req-b3debb8e-1ba1-4dae-99dc-018421ddb183'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:16.639385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88923ed488964be98362489d41fe1875', 'request_id': 'req-b3debb8e-1ba1-4dae-99dc-018421ddb183'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:16.762105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88923ed488964be98362489d41fe1875', 'request_id': 'req-b3debb8e-1ba1-4dae-99dc-018421ddb183'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:16.773614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] The token cache is being invalidate because project 88923ed488964be98362489d41fe1875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:11:16.774009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:16.774834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b3debb8e-1ba1-4dae-99dc-018421ddb183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:16.796633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1094/2193] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:11:16 2022] DELETE /identity/v3/projects/88923ed488964be98362489d41fe1875 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:11:19.740177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c98506-b1cc-4752-96c5-cae1a4235c16 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:19.740761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c98506-b1cc-4752-96c5-cae1a4235c16 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:19.742221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c98506-b1cc-4752-96c5-cae1a4235c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:19.769429 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-13c98506-b1cc-4752-96c5-cae1a4235c16 None None] MFA Rules not processed for user `4c7a6c1d08554db09d65394ec338f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:19.770784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-13c98506-b1cc-4752-96c5-cae1a4235c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:19.834129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636da1-f3b7-4f67-9863-55a9256b2ce7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:19.834641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636da1-f3b7-4f67-9863-55a9256b2ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:19.836329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636da1-f3b7-4f67-9863-55a9256b2ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:19.853286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1100/2194] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:19.858950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-76636da1-f3b7-4f67-9863-55a9256b2ce7 None None] MFA Rules not processed for user `6e4ebbf55d214dcc835e3f43876693bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:19.860156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-76636da1-f3b7-4f67-9863-55a9256b2ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:19.938280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1095/2195] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:11:22.228487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c98506-b1cc-4752-96c5-cae1a4235c16 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:22.229914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-13c98506-b1cc-4752-96c5-cae1a4235c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:22.275143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:22.301854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c98506-b1cc-4752-96c5-cae1a4235c16 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:22.306721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:22.312011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:22.312011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:22.312011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] PATH_INFO: `/v3/users/1d0af4ed24d0400ea94fa3edadac264b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:22.312011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0af4ed24d0400ea94fa3edadac264b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:22.312011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:22.553114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0af4ed24d0400ea94fa3edadac264b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:22.616524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0af4ed24d0400ea94fa3edadac264b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:22.625079 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0af4ed24d0400ea94fa3edadac264b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:22.625844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0af4ed24d0400ea94fa3edadac264b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:22.626540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0af4ed24d0400ea94fa3edadac264b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:22.647021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1101/2196] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:11:22 2022] DELETE /identity/v3/users/1d0af4ed24d0400ea94fa3edadac264b => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:11:22.651926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fd90f-11f7-44e5-8606-c05c29f7f7a1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:22.652934 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fd90f-11f7-44e5-8606-c05c29f7f7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:22.653209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fd90f-11f7-44e5-8606-c05c29f7f7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:22.682396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c06fd90f-11f7-44e5-8606-c05c29f7f7a1 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:22.683947 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c06fd90f-11f7-44e5-8606-c05c29f7f7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:22.747199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:22.797692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1096/2197] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:22.812407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:22.812407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:22.892362 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67766f4b-84f6-48b4-8675-82fe03da6504 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:22.896179 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efa791-3bcb-44a3-8626-24ca0db4b1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:22.897067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa791-3bcb-44a3-8626-24ca0db4b1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:22.898665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa791-3bcb-44a3-8626-24ca0db4b1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:22.898936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa791-3bcb-44a3-8626-24ca0db4b1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:22.911481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efa791-3bcb-44a3-8626-24ca0db4b1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:22.914024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efa791-3bcb-44a3-8626-24ca0db4b1c7 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:22.923243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1102/2198] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:11:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:11:23.423973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fd90f-11f7-44e5-8606-c05c29f7f7a1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:23.425452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c06fd90f-11f7-44e5-8606-c05c29f7f7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:23.439255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fd90f-11f7-44e5-8606-c05c29f7f7a1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:23.445449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:23.446077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:23.446345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:23.446609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] PATH_INFO: `/v3/projects/2cedb1ef78764683bdb6888501b18733` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:23.447731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cedb1ef78764683bdb6888501b18733)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:23.449997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:23.537028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cedb1ef78764683bdb6888501b18733', 'request_id': 'req-5ee038a0-702b-447c-820a-51c8101b5b61'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:23.543101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cedb1ef78764683bdb6888501b18733', 'request_id': 'req-5ee038a0-702b-447c-820a-51c8101b5b61'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:23.610770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cedb1ef78764683bdb6888501b18733', 'request_id': 'req-5ee038a0-702b-447c-820a-51c8101b5b61'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:23.622762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] The token cache is being invalidate because project 2cedb1ef78764683bdb6888501b18733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:11:23.623050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:23.623676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5ee038a0-702b-447c-820a-51c8101b5b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:23.625184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1097/2199] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:11:23 2022] DELETE /identity/v3/projects/2cedb1ef78764683bdb6888501b18733 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:11:23.630021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f590ede-c6ab-4769-800b-e78a80243a80 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:23.630502 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f590ede-c6ab-4769-800b-e78a80243a80 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:23.630594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f590ede-c6ab-4769-800b-e78a80243a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:23.649704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2f590ede-c6ab-4769-800b-e78a80243a80 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:23.650876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2f590ede-c6ab-4769-800b-e78a80243a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:23.700240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:23.709445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1103/2200] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:23.717155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b2a8f-8028-4f84-8543-ab12d7fbfe34 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:23.717635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b2a8f-8028-4f84-8543-ab12d7fbfe34 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:23.717752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b2a8f-8028-4f84-8543-ab12d7fbfe34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:23.737460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-637b2a8f-8028-4f84-8543-ab12d7fbfe34 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:23.738657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-637b2a8f-8028-4f84-8543-ab12d7fbfe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:23.747192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1098/2201] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:23.751416 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f590ede-c6ab-4769-800b-e78a80243a80 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:23.761144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f590ede-c6ab-4769-800b-e78a80243a80 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:23.765879 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be726b3-a34c-4d49-90c3-36975c0d5527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:23.766462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be726b3-a34c-4d49-90c3-36975c0d5527 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:23.766737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be726b3-a34c-4d49-90c3-36975c0d5527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:23.767006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be726b3-a34c-4d49-90c3-36975c0d5527 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:23.767646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be726b3-a34c-4d49-90c3-36975c0d5527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:11:23.767944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be726b3-a34c-4d49-90c3-36975c0d5527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:23.769704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be726b3-a34c-4d49-90c3-36975c0d5527 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:23.781749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1104/2202] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:11:23.785779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b2a8f-8028-4f84-8543-ab12d7fbfe34 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:23.810109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b2a8f-8028-4f84-8543-ab12d7fbfe34 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:23.814901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4938401-e113-409d-ad01-e25bdcd087ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:23.815600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4938401-e113-409d-ad01-e25bdcd087ac admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:23.816106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4938401-e113-409d-ad01-e25bdcd087ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:23.816610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4938401-e113-409d-ad01-e25bdcd087ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:23.817307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4938401-e113-409d-ad01-e25bdcd087ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:23.819274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4938401-e113-409d-ad01-e25bdcd087ac admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:23.895499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1099/2203] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:11:23 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:11:23.896651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be726b3-a34c-4d49-90c3-36975c0d5527 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:23.907864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be726b3-a34c-4d49-90c3-36975c0d5527 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:23.985077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c29b55-d806-4028-90cd-f57a8a5938cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:23.985618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c29b55-d806-4028-90cd-f57a8a5938cb admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:23.985890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c29b55-d806-4028-90cd-f57a8a5938cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:23.986153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c29b55-d806-4028-90cd-f57a8a5938cb admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:23.986894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c29b55-d806-4028-90cd-f57a8a5938cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:23.988870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c29b55-d806-4028-90cd-f57a8a5938cb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:24.075320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1105/2204] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:11:23 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:11:24.079452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4938401-e113-409d-ad01-e25bdcd087ac admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:24.089827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4938401-e113-409d-ad01-e25bdcd087ac admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:24.094552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9be06-6511-44e8-820c-9d54af05a53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:24.095183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9be06-6511-44e8-820c-9d54af05a53d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:24.095484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9be06-6511-44e8-820c-9d54af05a53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:24.095750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9be06-6511-44e8-820c-9d54af05a53d admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:24.096410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9be06-6511-44e8-820c-9d54af05a53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:24.098142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9be06-6511-44e8-820c-9d54af05a53d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:24.110342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1100/2205] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:11:24.114261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c29b55-d806-4028-90cd-f57a8a5938cb admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:24.125194 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c29b55-d806-4028-90cd-f57a8a5938cb admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:24.133910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5719f-1741-4d95-8901-0adc40de1e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:24.133910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5719f-1741-4d95-8901-0adc40de1e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:24.133910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5719f-1741-4d95-8901-0adc40de1e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:24.133910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5719f-1741-4d95-8901-0adc40de1e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:24.134368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5719f-1741-4d95-8901-0adc40de1e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:24.134368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5719f-1741-4d95-8901-0adc40de1e49 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:24.146821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1106/2206] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:11:24.149572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9be06-6511-44e8-820c-9d54af05a53d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:24.162117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9be06-6511-44e8-820c-9d54af05a53d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:24.167302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587924e-7134-4aea-895f-794eb9937c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:24.168369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587924e-7134-4aea-895f-794eb9937c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:24.168839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587924e-7134-4aea-895f-794eb9937c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:24.169290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587924e-7134-4aea-895f-794eb9937c70 admin admin] PATH_INFO: `/v3/projects/b7b03a7a803d4a8798ffd8cf567f6884/users/58ccd6873d37436387578320c1629ff0/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:24.185431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587924e-7134-4aea-895f-794eb9937c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b03a7a803d4a8798ffd8cf567f6884, user_id=58ccd6873d37436387578320c1629ff0, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:24.188099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587924e-7134-4aea-895f-794eb9937c70 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:24.244768 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1101/2207] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:11:24 2022] PUT /identity/v3/projects/b7b03a7a803d4a8798ffd8cf567f6884/users/58ccd6873d37436387578320c1629ff0/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:11:24.251485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa6e8f-7bf2-4b9e-adf8-59755bf57896 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:24.251773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa6e8f-7bf2-4b9e-adf8-59755bf57896 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:24.252044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa6e8f-7bf2-4b9e-adf8-59755bf57896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:24.282355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e0aa6e8f-7bf2-4b9e-adf8-59755bf57896 None None] MFA Rules not processed for user `58ccd6873d37436387578320c1629ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:24.283647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e0aa6e8f-7bf2-4b9e-adf8-59755bf57896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:24.370517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1107/2208] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:24.381627 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587924e-7134-4aea-895f-794eb9937c70 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:24.383426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3587924e-7134-4aea-895f-794eb9937c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:24.457793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587924e-7134-4aea-895f-794eb9937c70 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:24.462445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287571-26fb-48e0-b006-dc16e3e69b58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:24.463051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287571-26fb-48e0-b006-dc16e3e69b58 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:24.463429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287571-26fb-48e0-b006-dc16e3e69b58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:24.463792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287571-26fb-48e0-b006-dc16e3e69b58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:24.472716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287571-26fb-48e0-b006-dc16e3e69b58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:24.474136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287571-26fb-48e0-b006-dc16e3e69b58 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:24.485657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1102/2209] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:11:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:11:26.383501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3e27b-3758-41d8-bd5b-24634f885025 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:26.383974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3e27b-3758-41d8-bd5b-24634f885025 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:26.384326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3e27b-3758-41d8-bd5b-24634f885025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:26.412841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e6e3e27b-3758-41d8-bd5b-24634f885025 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:26.414621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e6e3e27b-3758-41d8-bd5b-24634f885025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:26.475069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:26.506405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287571-26fb-48e0-b006-dc16e3e69b58 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:26.508401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-df287571-26fb-48e0-b006-dc16e3e69b58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:26.509790 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1108/2210] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:11:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:26.517028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:26.591306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287571-26fb-48e0-b006-dc16e3e69b58 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:26.602766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154956c-9d26-45f3-b78b-a41c83ac7318 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:26.604238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154956c-9d26-45f3-b78b-a41c83ac7318 service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:26.604608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154956c-9d26-45f3-b78b-a41c83ac7318 service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:26.604948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154956c-9d26-45f3-b78b-a41c83ac7318 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:26.619911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154956c-9d26-45f3-b78b-a41c83ac7318 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:26.622450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154956c-9d26-45f3-b78b-a41c83ac7318 service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:26.634793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1103/2211] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:11:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 18:11:27.832395 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3e27b-3758-41d8-bd5b-24634f885025 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:28.012246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3e27b-3758-41d8-bd5b-24634f885025 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:28.017182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:28.018049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:28.018515 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:28.018993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] PATH_INFO: `/v3/users/58ccd6873d37436387578320c1629ff0` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:28.020350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ccd6873d37436387578320c1629ff0)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:28.022564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:28.208798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ccd6873d37436387578320c1629ff0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:28.302618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ccd6873d37436387578320c1629ff0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:28.324770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ccd6873d37436387578320c1629ff0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:28.333193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ccd6873d37436387578320c1629ff0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:28.340116 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ccd6873d37436387578320c1629ff0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:28.399447 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1109/2212] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:11:27 2022] DELETE /identity/v3/users/58ccd6873d37436387578320c1629ff0 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:11:28.403796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c050185-3b58-48ad-9d72-53dee573d435 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:28.404883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c050185-3b58-48ad-9d72-53dee573d435 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:28.405405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c050185-3b58-48ad-9d72-53dee573d435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:28.437821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3c050185-3b58-48ad-9d72-53dee573d435 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:28.438972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3c050185-3b58-48ad-9d72-53dee573d435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:28.516998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1104/2213] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:11:28.528260 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:28.530089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:28.626217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244d480-7abb-4d69-896f-875d0c64f279 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:28.629846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95157a5f-2e02-4515-b7d7-8c7fd41fea91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:28.630450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95157a5f-2e02-4515-b7d7-8c7fd41fea91 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:28.630772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95157a5f-2e02-4515-b7d7-8c7fd41fea91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:28.631003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95157a5f-2e02-4515-b7d7-8c7fd41fea91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:28.653146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95157a5f-2e02-4515-b7d7-8c7fd41fea91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:28.654610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95157a5f-2e02-4515-b7d7-8c7fd41fea91 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:28.689734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1110/2214] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:11:29.342868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c050185-3b58-48ad-9d72-53dee573d435 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:29.344404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3c050185-3b58-48ad-9d72-53dee573d435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:29.362308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c050185-3b58-48ad-9d72-53dee573d435 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:29.370361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:29.371051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:29.371328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:29.371688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] PATH_INFO: `/v3/projects/b7b03a7a803d4a8798ffd8cf567f6884` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:29.373856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7b03a7a803d4a8798ffd8cf567f6884)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:29.377820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:29.557858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7b03a7a803d4a8798ffd8cf567f6884', 'request_id': 'req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:29.570896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7b03a7a803d4a8798ffd8cf567f6884', 'request_id': 'req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:29.629433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7b03a7a803d4a8798ffd8cf567f6884', 'request_id': 'req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:29.645347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] The token cache is being invalidate because project b7b03a7a803d4a8798ffd8cf567f6884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:11:29.646093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:29.647968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-87a135d0-bfe3-41e6-818c-2a71d10b5a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:11:29.650599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1105/2215] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:11:29 2022] DELETE /identity/v3/projects/b7b03a7a803d4a8798ffd8cf567f6884 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:11:29.655517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32333730-6c01-4e75-a8c0-08d798252f30 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:29.656126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32333730-6c01-4e75-a8c0-08d798252f30 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:29.656126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32333730-6c01-4e75-a8c0-08d798252f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:29.695122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-32333730-6c01-4e75-a8c0-08d798252f30 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:29.696365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-32333730-6c01-4e75-a8c0-08d798252f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:29.732779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:29.834743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1111/2216] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:29.836876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b472-ef20-436d-93ab-a3385b0cd047 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:29.837836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b472-ef20-436d-93ab-a3385b0cd047 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:29.837836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b472-ef20-436d-93ab-a3385b0cd047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:29.880189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8e01b472-ef20-436d-93ab-a3385b0cd047 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:29.882294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8e01b472-ef20-436d-93ab-a3385b0cd047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:29.892123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1106/2217] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:29.896325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32333730-6c01-4e75-a8c0-08d798252f30 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:29.915636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32333730-6c01-4e75-a8c0-08d798252f30 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:29.922872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73f635-ec02-4c9c-be54-3ed01f80e750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:29.923499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73f635-ec02-4c9c-be54-3ed01f80e750 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:29.923777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73f635-ec02-4c9c-be54-3ed01f80e750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:29.924044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73f635-ec02-4c9c-be54-3ed01f80e750 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:29.924949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73f635-ec02-4c9c-be54-3ed01f80e750 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:11:29.927559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73f635-ec02-4c9c-be54-3ed01f80e750 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:29.929292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73f635-ec02-4c9c-be54-3ed01f80e750 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:29.961734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1112/2218] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:11:29.962468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01b472-ef20-436d-93ab-a3385b0cd047 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:29.973061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01b472-ef20-436d-93ab-a3385b0cd047 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:29.980360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a3dd-329e-4dc1-9b36-4b7e685a7fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:29.981496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a3dd-329e-4dc1-9b36-4b7e685a7fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:29.982293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a3dd-329e-4dc1-9b36-4b7e685a7fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:29.982797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a3dd-329e-4dc1-9b36-4b7e685a7fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:29.983661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333a3dd-329e-4dc1-9b36-4b7e685a7fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:29.985944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333a3dd-329e-4dc1-9b36-4b7e685a7fd9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:29.998809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:30.071259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1107/2219] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:11:29 2022] POST /identity/v3/projects => generated 361 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:11:30.076451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73f635-ec02-4c9c-be54-3ed01f80e750 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:30.088231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73f635-ec02-4c9c-be54-3ed01f80e750 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:30.173421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f05b8c-6452-4d1b-b7a0-f6db712f83af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:30.173991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f05b8c-6452-4d1b-b7a0-f6db712f83af admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:30.174558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f05b8c-6452-4d1b-b7a0-f6db712f83af admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:30.174835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f05b8c-6452-4d1b-b7a0-f6db712f83af admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:30.178152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f05b8c-6452-4d1b-b7a0-f6db712f83af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:30.178152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f05b8c-6452-4d1b-b7a0-f6db712f83af admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:30.267755 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1113/2220] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:11:30 2022] POST /identity/v3/users => generated 281 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:11:30.268592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a3dd-329e-4dc1-9b36-4b7e685a7fd9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:30.284037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a3dd-329e-4dc1-9b36-4b7e685a7fd9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:30.296736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d69029-7bd7-4155-a9bd-fd4eed4ea9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:30.298667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d69029-7bd7-4155-a9bd-fd4eed4ea9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:30.299051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d69029-7bd7-4155-a9bd-fd4eed4ea9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:30.299421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d69029-7bd7-4155-a9bd-fd4eed4ea9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:30.300182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d69029-7bd7-4155-a9bd-fd4eed4ea9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:30.303071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d69029-7bd7-4155-a9bd-fd4eed4ea9ee admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:30.316107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1108/2221] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:11:30.322255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f05b8c-6452-4d1b-b7a0-f6db712f83af admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:30.339429 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f05b8c-6452-4d1b-b7a0-f6db712f83af admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:30.344129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90d404-f82a-47a2-96aa-98d3ab8b89af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:30.344796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90d404-f82a-47a2-96aa-98d3ab8b89af admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:30.345071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90d404-f82a-47a2-96aa-98d3ab8b89af admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:30.345478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90d404-f82a-47a2-96aa-98d3ab8b89af admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:30.346319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90d404-f82a-47a2-96aa-98d3ab8b89af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:30.347924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90d404-f82a-47a2-96aa-98d3ab8b89af admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:30.362252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1114/2222] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:11:30.371509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d69029-7bd7-4155-a9bd-fd4eed4ea9ee admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:30.386167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d69029-7bd7-4155-a9bd-fd4eed4ea9ee admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:30.398538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcb7e2-2cf2-4da3-833f-ab28ce1c7215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:30.399087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcb7e2-2cf2-4da3-833f-ab28ce1c7215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:30.399360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcb7e2-2cf2-4da3-833f-ab28ce1c7215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:30.399629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcb7e2-2cf2-4da3-833f-ab28ce1c7215 admin admin] PATH_INFO: `/v3/projects/1bb86067cd2d48348affd0cc181e9e50/users/30491c6e94194304a5a9dbf42d8ef531/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:30.424438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fcb7e2-2cf2-4da3-833f-ab28ce1c7215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb86067cd2d48348affd0cc181e9e50, user_id=30491c6e94194304a5a9dbf42d8ef531, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:30.427289 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fcb7e2-2cf2-4da3-833f-ab28ce1c7215 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:30.480630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1109/2223] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:11:30 2022] PUT /identity/v3/projects/1bb86067cd2d48348affd0cc181e9e50/users/30491c6e94194304a5a9dbf42d8ef531/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:11:30.491828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680e961-acdc-4661-a607-a36e77063c20 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:30.492732 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680e961-acdc-4661-a607-a36e77063c20 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:30.493284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680e961-acdc-4661-a607-a36e77063c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:30.561694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-5680e961-acdc-4661-a607-a36e77063c20 None None] MFA Rules not processed for user `30491c6e94194304a5a9dbf42d8ef531`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:30.566335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5680e961-acdc-4661-a607-a36e77063c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:30.729339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1115/2224] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:11:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:30.749453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcb7e2-2cf2-4da3-833f-ab28ce1c7215 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:30.751838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-38fcb7e2-2cf2-4da3-833f-ab28ce1c7215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:30.853701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcb7e2-2cf2-4da3-833f-ab28ce1c7215 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:30.870094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a7d19-78f9-4c58-a9f0-df1de1764af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:30.871146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a7d19-78f9-4c58-a9f0-df1de1764af1 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:30.871611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a7d19-78f9-4c58-a9f0-df1de1764af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:30.871943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a7d19-78f9-4c58-a9f0-df1de1764af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:30.889339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a7d19-78f9-4c58-a9f0-df1de1764af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:30.891635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a7d19-78f9-4c58-a9f0-df1de1764af1 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:30.920534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1110/2225] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:11:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:11:39.487412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a39a3-d2bd-477f-91ef-ab7de84b93df None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:39.488003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a39a3-d2bd-477f-91ef-ab7de84b93df None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:39.488003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a39a3-d2bd-477f-91ef-ab7de84b93df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:39.509364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7f0a39a3-d2bd-477f-91ef-ab7de84b93df None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:11:39.510583 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7f0a39a3-d2bd-477f-91ef-ab7de84b93df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:39.521108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:39.572920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1116/2226] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:11:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:11:39.586414 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a7d19-78f9-4c58-a9f0-df1de1764af1 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:39.587861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-305a7d19-78f9-4c58-a9f0-df1de1764af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:39.664792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a7d19-78f9-4c58-a9f0-df1de1764af1 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:39.670055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b506c84-8cd2-418a-a11a-713710404087 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:39.670668 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b506c84-8cd2-418a-a11a-713710404087 service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:39.670946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b506c84-8cd2-418a-a11a-713710404087 service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:39.671244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b506c84-8cd2-418a-a11a-713710404087 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:39.681030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b506c84-8cd2-418a-a11a-713710404087 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:39.682838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b506c84-8cd2-418a-a11a-713710404087 service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:39.692451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1111/2227] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:11:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:11:41.329679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a39a3-d2bd-477f-91ef-ab7de84b93df None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:11:41.331182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7f0a39a3-d2bd-477f-91ef-ab7de84b93df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:41.416945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a39a3-d2bd-477f-91ef-ab7de84b93df None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:11:41.420939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024afc17-8e50-4429-b83d-cb3e858183f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:11:41.424358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024afc17-8e50-4429-b83d-cb3e858183f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:11:41.424358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024afc17-8e50-4429-b83d-cb3e858183f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:11:41.424358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024afc17-8e50-4429-b83d-cb3e858183f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:11:41.424358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-024afc17-8e50-4429-b83d-cb3e858183f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:11:41.524928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024afc17-8e50-4429-b83d-cb3e858183f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:11:41.526374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024afc17-8e50-4429-b83d-cb3e858183f9 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:11:41.549529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1117/2228] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:11:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:11:49.688479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=77,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:11:51.533063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=68 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:12:00.799161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e14b1-7c78-4de8-bc25-e493241c2581 None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:00.799739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e14b1-7c78-4de8-bc25-e493241c2581 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:00.799739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e14b1-7c78-4de8-bc25-e493241c2581 None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:00.801435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1112/2229] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:12:00.811147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e382e8-feb5-4643-a7c8-5738c6ab1a92 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:00.811622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e382e8-feb5-4643-a7c8-5738c6ab1a92 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:00.812015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e382e8-feb5-4643-a7c8-5738c6ab1a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:00.839137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-76e382e8-feb5-4643-a7c8-5738c6ab1a92 None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:00.840688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-76e382e8-feb5-4643-a7c8-5738c6ab1a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:00.927220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1118/2230] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:12:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:12:00.944297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e14b1-7c78-4de8-bc25-e493241c2581 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:00.946879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-971e14b1-7c78-4de8-bc25-e493241c2581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:01.016449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e14b1-7c78-4de8-bc25-e493241c2581 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:01.021259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034087b-7a5b-46bb-98cf-524d2e54426b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e95d295337b4472be946d252ac3bd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:01.022051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034087b-7a5b-46bb-98cf-524d2e54426b service glance] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:01.022472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034087b-7a5b-46bb-98cf-524d2e54426b service glance] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:01.022882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034087b-7a5b-46bb-98cf-524d2e54426b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:01.034628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034087b-7a5b-46bb-98cf-524d2e54426b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:01.036060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034087b-7a5b-46bb-98cf-524d2e54426b service glance] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:01.048276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1113/2231] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:12:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:12:01.266295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a0a5b-6fbe-408e-9c58-0a4ff5297e69 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:01.266993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a0a5b-6fbe-408e-9c58-0a4ff5297e69 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:01.266993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a0a5b-6fbe-408e-9c58-0a4ff5297e69 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:01.268723 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1119/2232] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:12:01.275262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62d79b-aed6-41b6-a5b9-b37b07931c21 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:01.279169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62d79b-aed6-41b6-a5b9-b37b07931c21 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:01.280654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62d79b-aed6-41b6-a5b9-b37b07931c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:01.303807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ed62d79b-aed6-41b6-a5b9-b37b07931c21 None None] MFA Rules not processed for user `95f02466042946ee96e37c5adc27b01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:01.305242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ed62d79b-aed6-41b6-a5b9-b37b07931c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:01.320596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1114/2233] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:12:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:01.327707 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a0a5b-6fbe-408e-9c58-0a4ff5297e69 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:01.340729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a0a5b-6fbe-408e-9c58-0a4ff5297e69 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:01.344457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe10f3-603f-4a6f-8e63-7967b67622aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:01.345079 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe10f3-603f-4a6f-8e63-7967b67622aa service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:01.345352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe10f3-603f-4a6f-8e63-7967b67622aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:01.345619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe10f3-603f-4a6f-8e63-7967b67622aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:01.354965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe10f3-603f-4a6f-8e63-7967b67622aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:01.356407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe10f3-603f-4a6f-8e63-7967b67622aa service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:01.365465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1120/2234] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:12:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:12:01.470364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c176b5-0c4b-4ed4-88d1-96d6567367bc None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:01.471238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c176b5-0c4b-4ed4-88d1-96d6567367bc None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:01.471527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c176b5-0c4b-4ed4-88d1-96d6567367bc None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:01.473319 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1115/2235] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:12:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:12:01.480980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d9d79-5e59-4478-ab31-24a1cbfc93f9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:01.481428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d9d79-5e59-4478-ab31-24a1cbfc93f9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:01.481776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d9d79-5e59-4478-ab31-24a1cbfc93f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:01.504916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b45d9d79-5e59-4478-ab31-24a1cbfc93f9 None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:01.506355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b45d9d79-5e59-4478-ab31-24a1cbfc93f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:01.517149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1121/2236] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:12:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:01.528527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c176b5-0c4b-4ed4-88d1-96d6567367bc None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:01.530349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-62c176b5-0c4b-4ed4-88d1-96d6567367bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:01.552004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c176b5-0c4b-4ed4-88d1-96d6567367bc None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:01.555735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63d303-39a7-4bb8-b91f-0b9e1dcc416a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:01.556196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63d303-39a7-4bb8-b91f-0b9e1dcc416a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:01.556472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63d303-39a7-4bb8-b91f-0b9e1dcc416a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:01.556778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63d303-39a7-4bb8-b91f-0b9e1dcc416a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:01.567785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63d303-39a7-4bb8-b91f-0b9e1dcc416a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:01.569449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63d303-39a7-4bb8-b91f-0b9e1dcc416a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:01.582417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1116/2237] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:12:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:12:08.813302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d9d79-5e59-4478-ab31-24a1cbfc93f9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:08.823221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d9d79-5e59-4478-ab31-24a1cbfc93f9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:08.827053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edb6cf-f9d0-44d0-b54b-5191ee8fb808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:08.827641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edb6cf-f9d0-44d0-b54b-5191ee8fb808 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:08.827918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edb6cf-f9d0-44d0-b54b-5191ee8fb808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:08.828181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edb6cf-f9d0-44d0-b54b-5191ee8fb808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:08.830036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d0edb6cf-f9d0-44d0-b54b-5191ee8fb808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:08.892768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edb6cf-f9d0-44d0-b54b-5191ee8fb808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:08.894192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edb6cf-f9d0-44d0-b54b-5191ee8fb808 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:08.903500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1122/2238] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:12:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:12:09.747194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63d303-39a7-4bb8-b91f-0b9e1dcc416a service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:09.748663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bb63d303-39a7-4bb8-b91f-0b9e1dcc416a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:09.819834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63d303-39a7-4bb8-b91f-0b9e1dcc416a service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:09.825191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c059f9-8f8f-46fc-9560-da741beaee3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:09.825789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c059f9-8f8f-46fc-9560-da741beaee3d service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:09.826145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c059f9-8f8f-46fc-9560-da741beaee3d service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:09.826365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c059f9-8f8f-46fc-9560-da741beaee3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:09.828091 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d0c059f9-8f8f-46fc-9560-da741beaee3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:09.840725 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c059f9-8f8f-46fc-9560-da741beaee3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:09.842392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c059f9-8f8f-46fc-9560-da741beaee3d service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:09.853484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1117/2239] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:12:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:12:12.395247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeb8ed-ad61-4531-bb76-c023334004a5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:12.395797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeb8ed-ad61-4531-bb76-c023334004a5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:12.395797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeb8ed-ad61-4531-bb76-c023334004a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:12.418073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1edeb8ed-ad61-4531-bb76-c023334004a5 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:12.419394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1edeb8ed-ad61-4531-bb76-c023334004a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:12.428523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1123/2240] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:12:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:12.452488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c059f9-8f8f-46fc-9560-da741beaee3d service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:12.464565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c059f9-8f8f-46fc-9560-da741beaee3d service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:12.470107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c362881-507a-4a7a-a5f0-95bb1900e230 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:12.470686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c362881-507a-4a7a-a5f0-95bb1900e230 service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:12.470955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c362881-507a-4a7a-a5f0-95bb1900e230 service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:12.477378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c362881-507a-4a7a-a5f0-95bb1900e230 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:12.480133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c362881-507a-4a7a-a5f0-95bb1900e230 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:12.481743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c362881-507a-4a7a-a5f0-95bb1900e230 service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:12.489863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1118/2241] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:12:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:12:19.281123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeb8ed-ad61-4531-bb76-c023334004a5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:19.357468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeb8ed-ad61-4531-bb76-c023334004a5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:19.362147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:19.369495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:19.369495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:19.369495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] PATH_INFO: `/v3/users/30491c6e94194304a5a9dbf42d8ef531` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:19.369495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=30491c6e94194304a5a9dbf42d8ef531)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:19.369495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:19.369495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:12:19.802530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30491c6e94194304a5a9dbf42d8ef531'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:19.936420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30491c6e94194304a5a9dbf42d8ef531'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:19.944817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30491c6e94194304a5a9dbf42d8ef531'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:19.945613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30491c6e94194304a5a9dbf42d8ef531'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:19.946368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30491c6e94194304a5a9dbf42d8ef531'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:19.966133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1124/2242] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:12:19 2022] DELETE /identity/v3/users/30491c6e94194304a5a9dbf42d8ef531 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:12:19.970989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a93ff-ffa6-40e1-9fb1-619bf14c766b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:19.971493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a93ff-ffa6-40e1-9fb1-619bf14c766b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:19.971830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a93ff-ffa6-40e1-9fb1-619bf14c766b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:19.991546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8f8a93ff-ffa6-40e1-9fb1-619bf14c766b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:19.992787 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8f8a93ff-ffa6-40e1-9fb1-619bf14c766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:20.054304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:12:20.063227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1119/2243] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:20.073503 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:20.075850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:20.144323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a84d20-4352-41d0-bc9e-4ec3c86232bd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:20.148045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9cefb-3536-4136-a1e5-e982fe4c06d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:20.148778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9cefb-3536-4136-a1e5-e982fe4c06d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:20.149055 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9cefb-3536-4136-a1e5-e982fe4c06d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:20.149325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9cefb-3536-4136-a1e5-e982fe4c06d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:20.160621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9cefb-3536-4136-a1e5-e982fe4c06d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:20.162066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9cefb-3536-4136-a1e5-e982fe4c06d6 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:20.172593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1125/2244] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:12:20.865107 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a93ff-ffa6-40e1-9fb1-619bf14c766b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:20.867039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8f8a93ff-ffa6-40e1-9fb1-619bf14c766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:20.883808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a93ff-ffa6-40e1-9fb1-619bf14c766b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:20.896057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:20.896741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:20.897014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:20.897279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] PATH_INFO: `/v3/projects/1bb86067cd2d48348affd0cc181e9e50` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:20.898601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb86067cd2d48348affd0cc181e9e50)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:20.906213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:21.017489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb86067cd2d48348affd0cc181e9e50', 'request_id': 'req-6c91a163-2f35-403a-ae70-53bbc928316f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:21.023915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb86067cd2d48348affd0cc181e9e50', 'request_id': 'req-6c91a163-2f35-403a-ae70-53bbc928316f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:21.102561 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb86067cd2d48348affd0cc181e9e50', 'request_id': 'req-6c91a163-2f35-403a-ae70-53bbc928316f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:21.115298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] The token cache is being invalidate because project 1bb86067cd2d48348affd0cc181e9e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:12:21.115670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:21.116371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-6c91a163-2f35-403a-ae70-53bbc928316f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:21.118027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1120/2245] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:12:20 2022] DELETE /identity/v3/projects/1bb86067cd2d48348affd0cc181e9e50 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:12:21.123229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ab8c8-7c24-412e-823a-b74569fd441c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:21.123774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ab8c8-7c24-412e-823a-b74569fd441c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:21.123774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ab8c8-7c24-412e-823a-b74569fd441c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:21.143738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-834ab8c8-7c24-412e-823a-b74569fd441c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:21.145022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-834ab8c8-7c24-412e-823a-b74569fd441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:21.209399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1126/2246] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:21.217615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1aee8-eb6e-4bf1-9866-ea1e37fddad0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:21.217966 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1aee8-eb6e-4bf1-9866-ea1e37fddad0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:21.218268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1aee8-eb6e-4bf1-9866-ea1e37fddad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:21.237410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d3f1aee8-eb6e-4bf1-9866-ea1e37fddad0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:21.238588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d3f1aee8-eb6e-4bf1-9866-ea1e37fddad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:21.246323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1121/2247] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:21.250373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ab8c8-7c24-412e-823a-b74569fd441c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:21.260020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ab8c8-7c24-412e-823a-b74569fd441c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:21.265096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec0bb7-685e-49ef-85ce-38b8f0de4088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:21.265684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec0bb7-685e-49ef-85ce-38b8f0de4088 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:21.265958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec0bb7-685e-49ef-85ce-38b8f0de4088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:21.266225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec0bb7-685e-49ef-85ce-38b8f0de4088 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:21.266864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec0bb7-685e-49ef-85ce-38b8f0de4088 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:12:21.267159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec0bb7-685e-49ef-85ce-38b8f0de4088 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:21.268762 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec0bb7-685e-49ef-85ce-38b8f0de4088 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:21.281019 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1127/2248] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:12:21.285875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1aee8-eb6e-4bf1-9866-ea1e37fddad0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:21.296916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1aee8-eb6e-4bf1-9866-ea1e37fddad0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:21.302262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b288b-6c6e-4cc4-a663-f6abc5831455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:21.302937 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b288b-6c6e-4cc4-a663-f6abc5831455 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:21.303299 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b288b-6c6e-4cc4-a663-f6abc5831455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:21.303572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b288b-6c6e-4cc4-a663-f6abc5831455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:21.304227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b288b-6c6e-4cc4-a663-f6abc5831455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:21.306351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b288b-6c6e-4cc4-a663-f6abc5831455 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:21.408164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1122/2249] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:12:21 2022] POST /identity/v3/projects => generated 397 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:12:21.412123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec0bb7-685e-49ef-85ce-38b8f0de4088 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:21.422234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec0bb7-685e-49ef-85ce-38b8f0de4088 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:21.426773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:12:21.478953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa3b90-077b-4c65-ab63-429bb6ecf3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:21.479570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa3b90-077b-4c65-ab63-429bb6ecf3a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:21.479840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa3b90-077b-4c65-ab63-429bb6ecf3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:21.480673 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa3b90-077b-4c65-ab63-429bb6ecf3a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:21.481301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa3b90-077b-4c65-ab63-429bb6ecf3a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:21.483270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa3b90-077b-4c65-ab63-429bb6ecf3a2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:21.565067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1128/2250] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:12:21 2022] POST /identity/v3/users => generated 299 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:12:21.569437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b288b-6c6e-4cc4-a663-f6abc5831455 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:21.580387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b288b-6c6e-4cc4-a663-f6abc5831455 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:21.585224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02165a-2537-4814-8f6f-01cacbe25536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:21.585834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02165a-2537-4814-8f6f-01cacbe25536 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:21.586111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02165a-2537-4814-8f6f-01cacbe25536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:21.586376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02165a-2537-4814-8f6f-01cacbe25536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:21.587030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02165a-2537-4814-8f6f-01cacbe25536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:21.588816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02165a-2537-4814-8f6f-01cacbe25536 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:21.616215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1123/2251] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:12:21.617880 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa3b90-077b-4c65-ab63-429bb6ecf3a2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:21.628382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa3b90-077b-4c65-ab63-429bb6ecf3a2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:21.633303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60712d-c972-4fdc-8f3e-edaeb40f8773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:21.633922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60712d-c972-4fdc-8f3e-edaeb40f8773 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:21.634198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60712d-c972-4fdc-8f3e-edaeb40f8773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:21.634466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60712d-c972-4fdc-8f3e-edaeb40f8773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:21.635101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60712d-c972-4fdc-8f3e-edaeb40f8773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:21.636742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60712d-c972-4fdc-8f3e-edaeb40f8773 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:21.647368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1129/2252] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:12:21.651588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02165a-2537-4814-8f6f-01cacbe25536 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:21.662494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02165a-2537-4814-8f6f-01cacbe25536 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:21.667163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4eb37-f118-4b29-b0f8-77f2dfe3413e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:21.668369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4eb37-f118-4b29-b0f8-77f2dfe3413e admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:21.668674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4eb37-f118-4b29-b0f8-77f2dfe3413e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:21.668942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4eb37-f118-4b29-b0f8-77f2dfe3413e admin admin] PATH_INFO: `/v3/projects/4d0b65d4bd1848178a3daea0ee1640c3/users/ec45aa4e62fd4e0cafcbf1fd23f2af92/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:21.687199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4eb37-f118-4b29-b0f8-77f2dfe3413e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0b65d4bd1848178a3daea0ee1640c3, user_id=ec45aa4e62fd4e0cafcbf1fd23f2af92, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:21.690240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4eb37-f118-4b29-b0f8-77f2dfe3413e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:21.846108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1124/2253] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:12:21 2022] PUT /identity/v3/projects/4d0b65d4bd1848178a3daea0ee1640c3/users/ec45aa4e62fd4e0cafcbf1fd23f2af92/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:12:21.857326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81e77b-d1ad-4cfc-8039-f3b40d71267f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:21.857655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81e77b-d1ad-4cfc-8039-f3b40d71267f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:21.857927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81e77b-d1ad-4cfc-8039-f3b40d71267f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:21.890940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ae81e77b-d1ad-4cfc-8039-f3b40d71267f None None] MFA Rules not processed for user `ec45aa4e62fd4e0cafcbf1fd23f2af92`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:21.892373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ae81e77b-d1ad-4cfc-8039-f3b40d71267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:21.965025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1130/2254] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:12:21 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:21.977211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4eb37-f118-4b29-b0f8-77f2dfe3413e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:21.979276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cdf4eb37-f118-4b29-b0f8-77f2dfe3413e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:22.043385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4eb37-f118-4b29-b0f8-77f2dfe3413e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:22.047457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b83a7-5534-4d4f-970b-a058fc3127ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:22.048147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b83a7-5534-4d4f-970b-a058fc3127ae service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:22.048490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b83a7-5534-4d4f-970b-a058fc3127ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:22.048863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b83a7-5534-4d4f-970b-a058fc3127ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:22.058901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b83a7-5534-4d4f-970b-a058fc3127ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:22.062832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b83a7-5534-4d4f-970b-a058fc3127ae service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:22.069916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1125/2255] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:12:21 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:12:31.027352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81e77b-d1ad-4cfc-8039-f3b40d71267f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:31.104598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81e77b-d1ad-4cfc-8039-f3b40d71267f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:31.110836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:31.111460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:31.111729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:31.112043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] PATH_INFO: `/v3/users/ec45aa4e62fd4e0cafcbf1fd23f2af92` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:31.113188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec45aa4e62fd4e0cafcbf1fd23f2af92)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:31.115089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:31.137132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:12:31.327457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec45aa4e62fd4e0cafcbf1fd23f2af92'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:31.387903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec45aa4e62fd4e0cafcbf1fd23f2af92'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:31.395800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec45aa4e62fd4e0cafcbf1fd23f2af92'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:31.401453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec45aa4e62fd4e0cafcbf1fd23f2af92'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:31.401453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec45aa4e62fd4e0cafcbf1fd23f2af92'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:31.415299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1131/2256] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:12:31 2022] DELETE /identity/v3/users/ec45aa4e62fd4e0cafcbf1fd23f2af92 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:12:31.420717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a92ab0-d7d7-45c9-802c-4a93d0e6c12a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:31.421199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a92ab0-d7d7-45c9-802c-4a93d0e6c12a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:31.421298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a92ab0-d7d7-45c9-802c-4a93d0e6c12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:31.440139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e0a92ab0-d7d7-45c9-802c-4a93d0e6c12a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:31.441604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e0a92ab0-d7d7-45c9-802c-4a93d0e6c12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:31.458491 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:12:31.501798 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1126/2257] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:31.511820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:31.513239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:31.576486 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77e29b-bc66-49e8-ae00-307d0615cca5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:31.580188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd537581-7cf0-414b-a944-69fb0ae80f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:31.580825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd537581-7cf0-414b-a944-69fb0ae80f01 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:31.581107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd537581-7cf0-414b-a944-69fb0ae80f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:31.581374 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd537581-7cf0-414b-a944-69fb0ae80f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:31.590583 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd537581-7cf0-414b-a944-69fb0ae80f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:31.592028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd537581-7cf0-414b-a944-69fb0ae80f01 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:31.601963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1132/2258] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:12:32.183578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a92ab0-d7d7-45c9-802c-4a93d0e6c12a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:32.185177 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e0a92ab0-d7d7-45c9-802c-4a93d0e6c12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:32.203112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a92ab0-d7d7-45c9-802c-4a93d0e6c12a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:32.210273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:32.214467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:32.214467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:32.214467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] PATH_INFO: `/v3/projects/4d0b65d4bd1848178a3daea0ee1640c3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:32.214467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0b65d4bd1848178a3daea0ee1640c3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:32.215444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:32.343739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0b65d4bd1848178a3daea0ee1640c3', 'request_id': 'req-9ef5985f-bf17-4f84-af05-f1db6a833137'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:32.353834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0b65d4bd1848178a3daea0ee1640c3', 'request_id': 'req-9ef5985f-bf17-4f84-af05-f1db6a833137'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:32.434042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0b65d4bd1848178a3daea0ee1640c3', 'request_id': 'req-9ef5985f-bf17-4f84-af05-f1db6a833137'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:32.458698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] The token cache is being invalidate because project 4d0b65d4bd1848178a3daea0ee1640c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:12:32.459823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:32.460840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:12:32.462838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1127/2259] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:12:32 2022] DELETE /identity/v3/projects/4d0b65d4bd1848178a3daea0ee1640c3 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:12:41.597978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=8,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:12:42.451884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:12:44.618047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac0bb2-1ec8-4179-8fcc-108f9aaecde3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:44.618525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac0bb2-1ec8-4179-8fcc-108f9aaecde3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:44.618609 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac0bb2-1ec8-4179-8fcc-108f9aaecde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:44.641695 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-85ac0bb2-1ec8-4179-8fcc-108f9aaecde3 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:44.645892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-85ac0bb2-1ec8-4179-8fcc-108f9aaecde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:44.703550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1133/2260] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 18:12:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:44.716841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:12:44.720482 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:44.787898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5985f-bf17-4f84-af05-f1db6a833137 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:12:44.793564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60d3c5-58b8-4c6c-9a2f-d7b5d02a5b88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:12:44.794183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60d3c5-58b8-4c6c-9a2f-d7b5d02a5b88 service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:44.794453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60d3c5-58b8-4c6c-9a2f-d7b5d02a5b88 service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:44.794713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60d3c5-58b8-4c6c-9a2f-d7b5d02a5b88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:44.803244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60d3c5-58b8-4c6c-9a2f-d7b5d02a5b88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:12:44.806454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60d3c5-58b8-4c6c-9a2f-d7b5d02a5b88 service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:12:44.813772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1128/2261] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:12:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:12:49.653224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a08cee-7cf0-4a17-b2a3-85ec5cba0a0c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:12:49.660401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a08cee-7cf0-4a17-b2a3-85ec5cba0a0c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:12:49.660401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a08cee-7cf0-4a17-b2a3-85ec5cba0a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:12:49.674042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-31a08cee-7cf0-4a17-b2a3-85ec5cba0a0c None None] MFA Rules not processed for user `2d5e7d5c77084705b1f39406293ab585`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:12:49.675219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-31a08cee-7cf0-4a17-b2a3-85ec5cba0a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:12:49.729020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1134/2262] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:12:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:12:54.810371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=32 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:12:59.722429 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=50 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:13:35.196605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1317-c610-4a65-a826-c771c08a7a30 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:13:35.197121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1317-c610-4a65-a826-c771c08a7a30 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:13:35.197121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1317-c610-4a65-a826-c771c08a7a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:13:35.216588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-994e1317-c610-4a65-a826-c771c08a7a30 None None] MFA Rules not processed for user `4c7a6c1d08554db09d65394ec338f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:13:35.219826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-994e1317-c610-4a65-a826-c771c08a7a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:13:35.270654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1129/2263] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:13:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:13:38.573539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004ad99-9e33-4964-b434-083d7f25a447 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:13:38.574044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004ad99-9e33-4964-b434-083d7f25a447 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:13:38.574345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004ad99-9e33-4964-b434-083d7f25a447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:13:38.594335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7004ad99-9e33-4964-b434-083d7f25a447 None None] MFA Rules not processed for user `2ec53f935a6c41f2af67e9fd2e13e12c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:13:38.595523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7004ad99-9e33-4964-b434-083d7f25a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:13:38.657311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1135/2264] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:13:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:13:45.264813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=25 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:13:48.649029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=25 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:13:51.352744 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74ea2e-4340-491d-bbb7-794504622ade None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:13:51.353267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74ea2e-4340-491d-bbb7-794504622ade None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:13:51.353267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74ea2e-4340-491d-bbb7-794504622ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:13:51.373256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7d74ea2e-4340-491d-bbb7-794504622ade None None] MFA Rules not processed for user `caf8d88ca4da4515bd9f1bec64da3375`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:13:51.374461 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7d74ea2e-4340-491d-bbb7-794504622ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:13:51.431941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1130/2265] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:13:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:13:51.469714 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004ad99-9e33-4964-b434-083d7f25a447 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:13:51.471154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7004ad99-9e33-4964-b434-083d7f25a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:13:51.529892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004ad99-9e33-4964-b434-083d7f25a447 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:13:51.533391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f60c8-24fe-4747-9e8a-d43382ebc052 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e95d295337b4472be946d252ac3bd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:13:51.533990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f60c8-24fe-4747-9e8a-d43382ebc052 service glance] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:13:51.534264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f60c8-24fe-4747-9e8a-d43382ebc052 service glance] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:13:51.534531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f60c8-24fe-4747-9e8a-d43382ebc052 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:13:51.536243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-722f60c8-24fe-4747-9e8a-d43382ebc052 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:13:51.599340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74ea2e-4340-491d-bbb7-794504622ade None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:13:51.600788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7d74ea2e-4340-491d-bbb7-794504622ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:13:51.611047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f60c8-24fe-4747-9e8a-d43382ebc052 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:13:51.612745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f60c8-24fe-4747-9e8a-d43382ebc052 service glance] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:13:51.626600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1136/2266] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:13:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:13:51.667416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74ea2e-4340-491d-bbb7-794504622ade None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:13:51.672055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b5a26-fd1e-4e50-86e6-65228ac2af6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:13:51.673033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b5a26-fd1e-4e50-86e6-65228ac2af6f service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:13:51.673251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b5a26-fd1e-4e50-86e6-65228ac2af6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:13:51.673473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b5a26-fd1e-4e50-86e6-65228ac2af6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:13:51.675598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e44b5a26-fd1e-4e50-86e6-65228ac2af6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:13:51.692158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b5a26-fd1e-4e50-86e6-65228ac2af6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:13:51.693655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b5a26-fd1e-4e50-86e6-65228ac2af6f service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:13:51.706249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1131/2267] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:13:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:13:59.573477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b1a91-efcb-4266-816a-8b42d51fa2e1 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:13:59.574004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b1a91-efcb-4266-816a-8b42d51fa2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:13:59.574004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b1a91-efcb-4266-816a-8b42d51fa2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:13:59.600892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-de0b1a91-efcb-4266-816a-8b42d51fa2e1 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:13:59.602442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-de0b1a91-efcb-4266-816a-8b42d51fa2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:13:59.610412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1137/2268] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 18:13:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:13:59.753796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b5a26-fd1e-4e50-86e6-65228ac2af6f service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:13:59.766278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b5a26-fd1e-4e50-86e6-65228ac2af6f service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:13:59.771340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e35687-3a01-4a0e-8ec6-adb00ada06bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:13:59.773608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35687-3a01-4a0e-8ec6-adb00ada06bd service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:13:59.773608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35687-3a01-4a0e-8ec6-adb00ada06bd service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:13:59.773608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35687-3a01-4a0e-8ec6-adb00ada06bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:13:59.780606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e35687-3a01-4a0e-8ec6-adb00ada06bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:13:59.782229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e35687-3a01-4a0e-8ec6-adb00ada06bd service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:13:59.790901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1132/2269] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:13:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:14:02.568871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b1a91-efcb-4266-816a-8b42d51fa2e1 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:02.570287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-de0b1a91-efcb-4266-816a-8b42d51fa2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:02.582550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b1a91-efcb-4266-816a-8b42d51fa2e1 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:02.586086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:02.586676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:02.586944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:02.587207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:02.588901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:02.600392 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:02.601899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:02.611125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1138/2270] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:06.454373 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e35687-3a01-4a0e-8ec6-adb00ada06bd service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:06.455957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f0e35687-3a01-4a0e-8ec6-adb00ada06bd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:06.472890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e35687-3a01-4a0e-8ec6-adb00ada06bd service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:06.478702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9be8f2-c8e2-4a49-a81e-86448208af33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:06.479440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9be8f2-c8e2-4a49-a81e-86448208af33 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:06.479825 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9be8f2-c8e2-4a49-a81e-86448208af33 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:06.480315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9be8f2-c8e2-4a49-a81e-86448208af33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:06.484423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6f9be8f2-c8e2-4a49-a81e-86448208af33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:06.497545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9be8f2-c8e2-4a49-a81e-86448208af33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:06.500035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9be8f2-c8e2-4a49-a81e-86448208af33 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:06.531683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1133/2271] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:10.089700 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:10.091254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:10.161962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757c109-4870-4a79-b6c1-7061ab24257b service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:10.167352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:10.167977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:10.168255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:10.168552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] PATH_INFO: `/v3/users/6e4ebbf55d214dcc835e3f43876693bb` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:10.169688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4ebbf55d214dcc835e3f43876693bb)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:10.172186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:10.195903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:10.362596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4ebbf55d214dcc835e3f43876693bb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:10.469438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4ebbf55d214dcc835e3f43876693bb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:10.478902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4ebbf55d214dcc835e3f43876693bb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:10.480050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4ebbf55d214dcc835e3f43876693bb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:10.480372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4ebbf55d214dcc835e3f43876693bb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:10.503182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1139/2272] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:14:10 2022] DELETE /identity/v3/users/6e4ebbf55d214dcc835e3f43876693bb => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:10.751450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9be8f2-c8e2-4a49-a81e-86448208af33 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:10.762922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9be8f2-c8e2-4a49-a81e-86448208af33 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:10.849258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:10.849890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:10.850203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:10.850906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] PATH_INFO: `/v3/projects/c3b1914ba55849e4a3a55a20ed50e7a1` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:10.852296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3b1914ba55849e4a3a55a20ed50e7a1)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:10.854449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:10.882703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:10.966151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3b1914ba55849e4a3a55a20ed50e7a1', 'request_id': 'req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:10.972010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3b1914ba55849e4a3a55a20ed50e7a1', 'request_id': 'req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:11.080543 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3b1914ba55849e4a3a55a20ed50e7a1', 'request_id': 'req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:11.093039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] The token cache is being invalidate because project c3b1914ba55849e4a3a55a20ed50e7a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:14:11.094437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:11.096605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:11.103064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1134/2273] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:14:10 2022] DELETE /identity/v3/projects/c3b1914ba55849e4a3a55a20ed50e7a1 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:11.103520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:11.105124 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:11.124264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f7e2b-5a30-4f2b-8afb-442eb900fe62 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:11.200368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:11.201041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:11.201312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:11.201959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] PATH_INFO: `/v3/users/5094cabe5ad9475da8a3e2b6cd808117` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:11.203379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5094cabe5ad9475da8a3e2b6cd808117)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:11.205729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:11.363925 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5094cabe5ad9475da8a3e2b6cd808117'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:11.457198 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5094cabe5ad9475da8a3e2b6cd808117'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:11.466575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5094cabe5ad9475da8a3e2b6cd808117'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:11.468484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5094cabe5ad9475da8a3e2b6cd808117'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:11.469264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d43a5f43-d08d-4e74-8307-eb42dc4241e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5094cabe5ad9475da8a3e2b6cd808117'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:11.494291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1140/2274] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:14:11 2022] DELETE /identity/v3/users/5094cabe5ad9475da8a3e2b6cd808117 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:11.766509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:11.768954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:11.800336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b619ad-29cf-4eb6-ab92-00a5cbd95ce8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:11.871995 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:11.872916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:11.874005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:11.874285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] PATH_INFO: `/v3/projects/df7079a696d54eb7bc8e9d4830077b37` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:11.875568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=df7079a696d54eb7bc8e9d4830077b37)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:11.877728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:12.020211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df7079a696d54eb7bc8e9d4830077b37', 'request_id': 'req-f65b3449-b020-4d78-9d59-7a7a68caab7a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:12.028432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df7079a696d54eb7bc8e9d4830077b37', 'request_id': 'req-f65b3449-b020-4d78-9d59-7a7a68caab7a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:12.114822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df7079a696d54eb7bc8e9d4830077b37', 'request_id': 'req-f65b3449-b020-4d78-9d59-7a7a68caab7a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:12.129537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] The token cache is being invalidate because project df7079a696d54eb7bc8e9d4830077b37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:14:12.129939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:12.130613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:12.132201 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1135/2275] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:14:11 2022] DELETE /identity/v3/projects/df7079a696d54eb7bc8e9d4830077b37 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:14.699506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0ae68-9ec5-4fbf-8f2a-b7544d791674 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:14.700008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0ae68-9ec5-4fbf-8f2a-b7544d791674 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:14.700106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0ae68-9ec5-4fbf-8f2a-b7544d791674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:14.720273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-60b0ae68-9ec5-4fbf-8f2a-b7544d791674 None None] MFA Rules not processed for user `2d5e7d5c77084705b1f39406293ab585`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:14.721553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-60b0ae68-9ec5-4fbf-8f2a-b7544d791674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:14.735126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:14.777497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1141/2276] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:14:15.082873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:15.084313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:15.130430 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:15.141957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b3449-b020-4d78-9d59-7a7a68caab7a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:15.145808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396e2df-8d36-4c27-beec-b7248e1329a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:15.146396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396e2df-8d36-4c27-beec-b7248e1329a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:15.146671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396e2df-8d36-4c27-beec-b7248e1329a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:15.146935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396e2df-8d36-4c27-beec-b7248e1329a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:15.148709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d396e2df-8d36-4c27-beec-b7248e1329a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:15.159384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396e2df-8d36-4c27-beec-b7248e1329a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:15.160849 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396e2df-8d36-4c27-beec-b7248e1329a6 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:15.169021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1136/2277] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:15.603655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0ae68-9ec5-4fbf-8f2a-b7544d791674 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:15.605040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-60b0ae68-9ec5-4fbf-8f2a-b7544d791674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:15.679823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0ae68-9ec5-4fbf-8f2a-b7544d791674 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:15.685376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2876244-2f15-4bff-b65f-4d41a74c43d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:15.685976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2876244-2f15-4bff-b65f-4d41a74c43d3 service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:15.686250 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2876244-2f15-4bff-b65f-4d41a74c43d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:15.686999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2876244-2f15-4bff-b65f-4d41a74c43d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:15.688765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f2876244-2f15-4bff-b65f-4d41a74c43d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:15.700058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2876244-2f15-4bff-b65f-4d41a74c43d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:15.701689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2876244-2f15-4bff-b65f-4d41a74c43d3 service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:15.712254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1142/2278] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:16.448058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396e2df-8d36-4c27-beec-b7248e1329a6 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:16.457708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396e2df-8d36-4c27-beec-b7248e1329a6 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:16.461905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9d183-a576-4a3c-b2bd-61359a04d96b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:16.462618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9d183-a576-4a3c-b2bd-61359a04d96b service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:16.462896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9d183-a576-4a3c-b2bd-61359a04d96b service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:16.463162 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9d183-a576-4a3c-b2bd-61359a04d96b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:16.465025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-61b9d183-a576-4a3c-b2bd-61359a04d96b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:16.475538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9d183-a576-4a3c-b2bd-61359a04d96b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:16.477000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9d183-a576-4a3c-b2bd-61359a04d96b service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:16.485713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1137/2279] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:20.673652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495208f-a95c-4981-9248-6da8b98f49f7 None None] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:20.674203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495208f-a95c-4981-9248-6da8b98f49f7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:20.674203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495208f-a95c-4981-9248-6da8b98f49f7 None None] PATH_INFO: `/` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:20.675684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1143/2280] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:14:20.681957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc76f0-9c69-4b77-a5d6-ef75f20fba60 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:20.682399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc76f0-9c69-4b77-a5d6-ef75f20fba60 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:20.682494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc76f0-9c69-4b77-a5d6-ef75f20fba60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:20.703400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e6bc76f0-9c69-4b77-a5d6-ef75f20fba60 None None] MFA Rules not processed for user `95f02466042946ee96e37c5adc27b01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:20.704685 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e6bc76f0-9c69-4b77-a5d6-ef75f20fba60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:20.713988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1138/2281] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:14:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:20.724462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495208f-a95c-4981-9248-6da8b98f49f7 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:20.726257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2495208f-a95c-4981-9248-6da8b98f49f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:20.740464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495208f-a95c-4981-9248-6da8b98f49f7 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:20.744056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a420206-9770-42eb-89dc-611249197de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:20.746636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a420206-9770-42eb-89dc-611249197de1 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:20.746636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a420206-9770-42eb-89dc-611249197de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:20.746636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a420206-9770-42eb-89dc-611249197de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:20.752445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a420206-9770-42eb-89dc-611249197de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:20.753914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a420206-9770-42eb-89dc-611249197de1 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:20.762210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1144/2282] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:20.851763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4592a5-bddb-4658-86e3-ac6abe209c5d None None] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:20.852062 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4592a5-bddb-4658-86e3-ac6abe209c5d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:20.852332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4592a5-bddb-4658-86e3-ac6abe209c5d None None] PATH_INFO: `/` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:20.853740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1139/2283] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 18:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 18:14:20.859317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d3d1b-502a-4bdb-aee6-840e8619e572 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:20.859597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d3d1b-502a-4bdb-aee6-840e8619e572 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:20.859866 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d3d1b-502a-4bdb-aee6-840e8619e572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:20.886619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6c6d3d1b-502a-4bdb-aee6-840e8619e572 None None] MFA Rules not processed for user `c496da229db34f90a4b3fdcfa582dca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:20.886619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6c6d3d1b-502a-4bdb-aee6-840e8619e572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:20.956321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1145/2284] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 18:14:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:14:20.970888 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4592a5-bddb-4658-86e3-ac6abe209c5d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:20.982193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4592a5-bddb-4658-86e3-ac6abe209c5d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:20.986395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15cc62-e531-495f-947a-e5a169d840c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:20.986987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15cc62-e531-495f-947a-e5a169d840c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:20.987271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15cc62-e531-495f-947a-e5a169d840c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:20.987550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15cc62-e531-495f-947a-e5a169d840c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:20.998437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15cc62-e531-495f-947a-e5a169d840c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:21.000278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15cc62-e531-495f-947a-e5a169d840c9 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:21.011672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1140/2285] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:21.583358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d3d1b-502a-4bdb-aee6-840e8619e572 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:21.584792 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6c6d3d1b-502a-4bdb-aee6-840e8619e572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:21.620125 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:21.645155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d3d1b-502a-4bdb-aee6-840e8619e572 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:21.650946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204347e-2ead-430d-81f1-10aac54d170b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:21.650946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204347e-2ead-430d-81f1-10aac54d170b service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:21.650946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204347e-2ead-430d-81f1-10aac54d170b service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:21.650946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204347e-2ead-430d-81f1-10aac54d170b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:21.651536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5204347e-2ead-430d-81f1-10aac54d170b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:21.713920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204347e-2ead-430d-81f1-10aac54d170b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:21.715491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204347e-2ead-430d-81f1-10aac54d170b service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:21.725169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1146/2286] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:21 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:30.873254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15cc62-e531-495f-947a-e5a169d840c9 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:30.875407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-df15cc62-e531-495f-947a-e5a169d840c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:30.891901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15cc62-e531-495f-947a-e5a169d840c9 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:30.898066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b1229-235a-452b-b550-279e8f77043f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:30.898679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b1229-235a-452b-b550-279e8f77043f service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:30.898957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b1229-235a-452b-b550-279e8f77043f service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:30.899226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b1229-235a-452b-b550-279e8f77043f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:30.902207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-145b1229-235a-452b-b550-279e8f77043f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:30.924912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b1229-235a-452b-b550-279e8f77043f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:30.924912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b1229-235a-452b-b550-279e8f77043f service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:30.939338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1141/2287] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:31.720751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=33 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:39.140756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204347e-2ead-430d-81f1-10aac54d170b service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:39.154193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204347e-2ead-430d-81f1-10aac54d170b service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:39.159959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b8eb8-ca8f-4c40-b0eb-2f5697792d70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:39.160730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b8eb8-ca8f-4c40-b0eb-2f5697792d70 service nova] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:39.161010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b8eb8-ca8f-4c40-b0eb-2f5697792d70 service nova] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:39.161338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b8eb8-ca8f-4c40-b0eb-2f5697792d70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:39.163072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-605b8eb8-ca8f-4c40-b0eb-2f5697792d70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:39.226062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b8eb8-ca8f-4c40-b0eb-2f5697792d70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:39.227667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b8eb8-ca8f-4c40-b0eb-2f5697792d70 service nova] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:39.238276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1147/2288] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:40.282960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b1229-235a-452b-b550-279e8f77043f service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:40.294407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b1229-235a-452b-b550-279e8f77043f service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:40.300640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d42986-519f-441e-923b-a85e07a0fb97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:40.301226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d42986-519f-441e-923b-a85e07a0fb97 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:40.301498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d42986-519f-441e-923b-a85e07a0fb97 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:40.301762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d42986-519f-441e-923b-a85e07a0fb97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:40.303509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c4d42986-519f-441e-923b-a85e07a0fb97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:40.315534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d42986-519f-441e-923b-a85e07a0fb97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:40.317136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d42986-519f-441e-923b-a85e07a0fb97 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:40.327481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1142/2289] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:47.700845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b8eb8-ca8f-4c40-b0eb-2f5697792d70 service nova] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:47.710424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b8eb8-ca8f-4c40-b0eb-2f5697792d70 service nova] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:47.713976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:47.714548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:47.714818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:47.715081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:47.716800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:47.781414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:47.782847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:47.792346 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1148/2290] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:50.323867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=34 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:51.145700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d42986-519f-441e-923b-a85e07a0fb97 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:51.157597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d42986-519f-441e-923b-a85e07a0fb97 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:51.162700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a2508-0ed7-4b0a-8e18-04e994c86c59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:51.163293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a2508-0ed7-4b0a-8e18-04e994c86c59 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:51.163567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a2508-0ed7-4b0a-8e18-04e994c86c59 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:51.163834 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a2508-0ed7-4b0a-8e18-04e994c86c59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:51.165976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-065a2508-0ed7-4b0a-8e18-04e994c86c59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:51.177532 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a2508-0ed7-4b0a-8e18-04e994c86c59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:51.179096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a2508-0ed7-4b0a-8e18-04e994c86c59 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:51.189494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1143/2291] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:51.276003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:51.277493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:51.290264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e3aed-a10d-454b-9de7-54117d3bdd19 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:51.294808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:51.295457 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:51.295734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:51.296000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] PATH_INFO: `/v3/users/4c7a6c1d08554db09d65394ec338f8b5` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:51.297131 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7a6c1d08554db09d65394ec338f8b5)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:51.303068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:51.500967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7a6c1d08554db09d65394ec338f8b5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:51.578822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7a6c1d08554db09d65394ec338f8b5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:51.589499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7a6c1d08554db09d65394ec338f8b5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:51.589499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7a6c1d08554db09d65394ec338f8b5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:51.589499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-48b90139-4bbf-40e2-950c-0d15f8a30b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7a6c1d08554db09d65394ec338f8b5'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:51.605285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1149/2292] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:14:51 2022] DELETE /identity/v3/users/4c7a6c1d08554db09d65394ec338f8b5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:51.836946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a2508-0ed7-4b0a-8e18-04e994c86c59 service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:51.849765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a2508-0ed7-4b0a-8e18-04e994c86c59 service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:51.907097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:51.907770 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:51.908045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:51.908317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] PATH_INFO: `/v3/projects/34b812a3f98747eeac35fbc27a342ab0` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:51.909509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34b812a3f98747eeac35fbc27a342ab0)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:51.911407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:51.994994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34b812a3f98747eeac35fbc27a342ab0', 'request_id': 'req-427cecee-67a2-4c62-a555-7a72a8e194d1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:52.003203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34b812a3f98747eeac35fbc27a342ab0', 'request_id': 'req-427cecee-67a2-4c62-a555-7a72a8e194d1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:52.051449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34b812a3f98747eeac35fbc27a342ab0', 'request_id': 'req-427cecee-67a2-4c62-a555-7a72a8e194d1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:52.063455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] The token cache is being invalidate because project 34b812a3f98747eeac35fbc27a342ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:14:52.063739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:52.064390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-427cecee-67a2-4c62-a555-7a72a8e194d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:52.065892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1144/2293] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:14:51 2022] DELETE /identity/v3/projects/34b812a3f98747eeac35fbc27a342ab0 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:52.070905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221a323-bfe4-462e-92b4-851e9ce6fc9a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.071192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221a323-bfe4-462e-92b4-851e9ce6fc9a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.071465 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221a323-bfe4-462e-92b4-851e9ce6fc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.080122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:52.091063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6221a323-bfe4-462e-92b4-851e9ce6fc9a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:52.092297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6221a323-bfe4-462e-92b4-851e9ce6fc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:52.150799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1150/2294] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:52.159004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0eb47-cc21-4814-b8d2-9696386fd634 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.159295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0eb47-cc21-4814-b8d2-9696386fd634 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.160643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0eb47-cc21-4814-b8d2-9696386fd634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.179153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-4df0eb47-cc21-4814-b8d2-9696386fd634 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:52.180348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4df0eb47-cc21-4814-b8d2-9696386fd634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:52.188205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1145/2295] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:52.192447 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221a323-bfe4-462e-92b4-851e9ce6fc9a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:52.202190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221a323-bfe4-462e-92b4-851e9ce6fc9a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:52.206895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce086df-5c17-43db-a46a-aa9037d8042c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:52.207475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce086df-5c17-43db-a46a-aa9037d8042c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.207749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce086df-5c17-43db-a46a-aa9037d8042c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.208016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce086df-5c17-43db-a46a-aa9037d8042c admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.208692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce086df-5c17-43db-a46a-aa9037d8042c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:14:52.208987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce086df-5c17-43db-a46a-aa9037d8042c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:52.210581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce086df-5c17-43db-a46a-aa9037d8042c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:52.222412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1151/2296] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:14:52.226521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0eb47-cc21-4814-b8d2-9696386fd634 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:52.236285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0eb47-cc21-4814-b8d2-9696386fd634 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:52.240925 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a367ab9-f0fb-44b6-a54c-244601f9f066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:52.241507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a367ab9-f0fb-44b6-a54c-244601f9f066 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.241781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a367ab9-f0fb-44b6-a54c-244601f9f066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.242046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a367ab9-f0fb-44b6-a54c-244601f9f066 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.242668 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a367ab9-f0fb-44b6-a54c-244601f9f066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:52.244562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a367ab9-f0fb-44b6-a54c-244601f9f066 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:52.321741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1146/2297] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:14:52 2022] POST /identity/v3/projects => generated 389 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:14:52.325429 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce086df-5c17-43db-a46a-aa9037d8042c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:52.336534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce086df-5c17-43db-a46a-aa9037d8042c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:52.399733 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451e9dd-fa63-4431-99f9-bd9754e1ff2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:52.400339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451e9dd-fa63-4431-99f9-bd9754e1ff2c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.400653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451e9dd-fa63-4431-99f9-bd9754e1ff2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.400922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451e9dd-fa63-4431-99f9-bd9754e1ff2c admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.401555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451e9dd-fa63-4431-99f9-bd9754e1ff2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:52.403475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451e9dd-fa63-4431-99f9-bd9754e1ff2c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:52.492039 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1152/2298] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:14:52 2022] POST /identity/v3/users => generated 295 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:14:52.496524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a367ab9-f0fb-44b6-a54c-244601f9f066 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:52.507306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a367ab9-f0fb-44b6-a54c-244601f9f066 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:52.511961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e0f51-6738-4287-ac59-8b78e8802ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:52.512581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e0f51-6738-4287-ac59-8b78e8802ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.512859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e0f51-6738-4287-ac59-8b78e8802ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.513130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e0f51-6738-4287-ac59-8b78e8802ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.513763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e0f51-6738-4287-ac59-8b78e8802ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:52.515478 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e0f51-6738-4287-ac59-8b78e8802ef3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:52.525554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1147/2299] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:14:52.529519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451e9dd-fa63-4431-99f9-bd9754e1ff2c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:52.539981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451e9dd-fa63-4431-99f9-bd9754e1ff2c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:52.544739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2dd20-999d-4257-a499-9411c7f95337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:52.545247 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2dd20-999d-4257-a499-9411c7f95337 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.546292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2dd20-999d-4257-a499-9411c7f95337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.546565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2dd20-999d-4257-a499-9411c7f95337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.547197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2dd20-999d-4257-a499-9411c7f95337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:52.548815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2dd20-999d-4257-a499-9411c7f95337 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:52.560483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1153/2300] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:14:52.564522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e0f51-6738-4287-ac59-8b78e8802ef3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:52.574484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e0f51-6738-4287-ac59-8b78e8802ef3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:52.579786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775478fb-f1b3-4619-925d-c97e204e31f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:52.580411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775478fb-f1b3-4619-925d-c97e204e31f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.580713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775478fb-f1b3-4619-925d-c97e204e31f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.584425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775478fb-f1b3-4619-925d-c97e204e31f9 admin admin] PATH_INFO: `/v3/projects/5fb89c4b9bc04bde91fb1bae662d5489/users/61a016eaf47c4ccfb1bc04b6a8d546dd/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.596284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775478fb-f1b3-4619-925d-c97e204e31f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb89c4b9bc04bde91fb1bae662d5489, user_id=61a016eaf47c4ccfb1bc04b6a8d546dd, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:52.598903 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775478fb-f1b3-4619-925d-c97e204e31f9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:52.664171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1148/2301] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:14:52 2022] PUT /identity/v3/projects/5fb89c4b9bc04bde91fb1bae662d5489/users/61a016eaf47c4ccfb1bc04b6a8d546dd/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:52.675499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b9214-7136-459f-9b1c-fa6e740a0de3 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.675785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b9214-7136-459f-9b1c-fa6e740a0de3 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.676054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b9214-7136-459f-9b1c-fa6e740a0de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.706238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-850b9214-7136-459f-9b1c-fa6e740a0de3 None None] MFA Rules not processed for user `61a016eaf47c4ccfb1bc04b6a8d546dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:52.707544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-850b9214-7136-459f-9b1c-fa6e740a0de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:52.771150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:52.774585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1154/2302] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:52.779193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775478fb-f1b3-4619-925d-c97e204e31f9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:52.793241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:52.841687 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775478fb-f1b3-4619-925d-c97e204e31f9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:52.847266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611e12-f72b-4d9b-b1a0-b3784335883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:52.847858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611e12-f72b-4d9b-b1a0-b3784335883b admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:52.848134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611e12-f72b-4d9b-b1a0-b3784335883b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:52.848402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611e12-f72b-4d9b-b1a0-b3784335883b admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:52.849054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67611e12-f72b-4d9b-b1a0-b3784335883b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:52.853097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67611e12-f72b-4d9b-b1a0-b3784335883b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:52.973503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1149/2303] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:14:52 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:14:52.977490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b9214-7136-459f-9b1c-fa6e740a0de3 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:52.988471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b9214-7136-459f-9b1c-fa6e740a0de3 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:53.043897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac9b5f-135f-4e52-9cce-dd4c7eba9aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:53.044418 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac9b5f-135f-4e52-9cce-dd4c7eba9aae admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:53.044742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac9b5f-135f-4e52-9cce-dd4c7eba9aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:53.045013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac9b5f-135f-4e52-9cce-dd4c7eba9aae admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:53.045740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac9b5f-135f-4e52-9cce-dd4c7eba9aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:53.047689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac9b5f-135f-4e52-9cce-dd4c7eba9aae admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:53.109092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1155/2304] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:14:52 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:14:53.113238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611e12-f72b-4d9b-b1a0-b3784335883b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:53.123406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611e12-f72b-4d9b-b1a0-b3784335883b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:53.128476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552833e-a292-40d3-8357-e52a53ad8bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:53.129083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552833e-a292-40d3-8357-e52a53ad8bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:53.129357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552833e-a292-40d3-8357-e52a53ad8bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:53.129618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552833e-a292-40d3-8357-e52a53ad8bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:53.130244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552833e-a292-40d3-8357-e52a53ad8bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:53.131826 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552833e-a292-40d3-8357-e52a53ad8bc8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:53.142719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1150/2305] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:14:53.146818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac9b5f-135f-4e52-9cce-dd4c7eba9aae admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:53.156719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac9b5f-135f-4e52-9cce-dd4c7eba9aae admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:53.161277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d269b4-59e7-4ade-8ef8-0da376302007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:53.161910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d269b4-59e7-4ade-8ef8-0da376302007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:53.162183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d269b4-59e7-4ade-8ef8-0da376302007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:53.162450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d269b4-59e7-4ade-8ef8-0da376302007 admin admin] PATH_INFO: `/v3/projects/68069624eec94d199e1a91d9401b63f2/users/eeb7debd3ed64db381a5c41e4474630f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:53.176719 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d269b4-59e7-4ade-8ef8-0da376302007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68069624eec94d199e1a91d9401b63f2, user_id=eeb7debd3ed64db381a5c41e4474630f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:53.179367 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d269b4-59e7-4ade-8ef8-0da376302007 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:53.246143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1156/2306] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:14:53 2022] PUT /identity/v3/projects/68069624eec94d199e1a91d9401b63f2/users/eeb7debd3ed64db381a5c41e4474630f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:53.250647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552833e-a292-40d3-8357-e52a53ad8bc8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:53.260548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552833e-a292-40d3-8357-e52a53ad8bc8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:53.316076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ac2bc-eaa9-4d7b-8650-d2b46b5d9a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:53.316821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ac2bc-eaa9-4d7b-8650-d2b46b5d9a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:53.317092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ac2bc-eaa9-4d7b-8650-d2b46b5d9a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:53.317357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ac2bc-eaa9-4d7b-8650-d2b46b5d9a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:53.317990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ac2bc-eaa9-4d7b-8650-d2b46b5d9a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:53.319556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ac2bc-eaa9-4d7b-8650-d2b46b5d9a45 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:53.329429 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1151/2307] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:14:53.334138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d269b4-59e7-4ade-8ef8-0da376302007 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:53.343904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d269b4-59e7-4ade-8ef8-0da376302007 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:53.348613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaa841-9229-48d4-a8f6-f526b40a6868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:53.349274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaa841-9229-48d4-a8f6-f526b40a6868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:53.349537 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaa841-9229-48d4-a8f6-f526b40a6868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:53.356027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaa841-9229-48d4-a8f6-f526b40a6868 admin admin] PATH_INFO: `/v3/projects/68069624eec94d199e1a91d9401b63f2/users/eeb7debd3ed64db381a5c41e4474630f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:53.356027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaa841-9229-48d4-a8f6-f526b40a6868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68069624eec94d199e1a91d9401b63f2, user_id=eeb7debd3ed64db381a5c41e4474630f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:53.356027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaa841-9229-48d4-a8f6-f526b40a6868 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:53.364799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1157/2308] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:14:53 2022] PUT /identity/v3/projects/68069624eec94d199e1a91d9401b63f2/users/eeb7debd3ed64db381a5c41e4474630f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:53.375282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a61d52-2363-473f-9ae2-15bca6638bf5 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:53.375568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a61d52-2363-473f-9ae2-15bca6638bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:53.375839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a61d52-2363-473f-9ae2-15bca6638bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:53.402897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-29a61d52-2363-473f-9ae2-15bca6638bf5 None None] MFA Rules not processed for user `eeb7debd3ed64db381a5c41e4474630f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:53.404083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-29a61d52-2363-473f-9ae2-15bca6638bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:53.470009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1152/2309] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:53.480576 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaa841-9229-48d4-a8f6-f526b40a6868 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:53.482318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-efcaa841-9229-48d4-a8f6-f526b40a6868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:53.537509 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaa841-9229-48d4-a8f6-f526b40a6868 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:53.541070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0240b-a9d6-4000-98f6-fd9b0c0b4640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:53.541647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0240b-a9d6-4000-98f6-fd9b0c0b4640 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:53.541918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0240b-a9d6-4000-98f6-fd9b0c0b4640 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:53.542179 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0240b-a9d6-4000-98f6-fd9b0c0b4640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:53.551303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0240b-a9d6-4000-98f6-fd9b0c0b4640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:53.552735 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0240b-a9d6-4000-98f6-fd9b0c0b4640 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:53.562338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1158/2310] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:53.696568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a61d52-2363-473f-9ae2-15bca6638bf5 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:53.706673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:53.764576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a61d52-2363-473f-9ae2-15bca6638bf5 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:53.769178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:53.769797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:53.770064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:53.770339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] PATH_INFO: `/v3/users/61a016eaf47c4ccfb1bc04b6a8d546dd` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:53.771595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a016eaf47c4ccfb1bc04b6a8d546dd)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:53.773575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:53.914053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a016eaf47c4ccfb1bc04b6a8d546dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:53.966284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a016eaf47c4ccfb1bc04b6a8d546dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:53.974583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a016eaf47c4ccfb1bc04b6a8d546dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:53.975234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a016eaf47c4ccfb1bc04b6a8d546dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:53.975888 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a016eaf47c4ccfb1bc04b6a8d546dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:53.993822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1153/2311] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:14:53 2022] DELETE /identity/v3/users/61a016eaf47c4ccfb1bc04b6a8d546dd => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:53.998223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d23fd9-ce51-4698-b230-a502bea8db77 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:53.998505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d23fd9-ce51-4698-b230-a502bea8db77 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:53.998775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d23fd9-ce51-4698-b230-a502bea8db77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:54.018453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-56d23fd9-ce51-4698-b230-a502bea8db77 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:54.019647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-56d23fd9-ce51-4698-b230-a502bea8db77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:54.069872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:54.081596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1159/2312] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:54.091424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:54.093434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:54.150797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b426ff-ea7a-4b29-86af-ffa5d0787642 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:54.155937 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8037b-b00d-4b52-bbf8-9310a0ab70c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:54.157039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8037b-b00d-4b52-bbf8-9310a0ab70c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:54.157533 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8037b-b00d-4b52-bbf8-9310a0ab70c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:54.158733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8037b-b00d-4b52-bbf8-9310a0ab70c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:54.168612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8037b-b00d-4b52-bbf8-9310a0ab70c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:54.170032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8037b-b00d-4b52-bbf8-9310a0ab70c4 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:54.179288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1154/2313] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:54.573100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d23fd9-ce51-4698-b230-a502bea8db77 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:54.574515 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-56d23fd9-ce51-4698-b230-a502bea8db77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:54.589518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d23fd9-ce51-4698-b230-a502bea8db77 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:54.594415 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:54.595153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:54.595428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:54.595691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] PATH_INFO: `/v3/projects/5fb89c4b9bc04bde91fb1bae662d5489` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:54.596840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fb89c4b9bc04bde91fb1bae662d5489)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:54.598734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:54.702372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fb89c4b9bc04bde91fb1bae662d5489', 'request_id': 'req-0b940211-743b-4e54-a50f-25ccadc4cd2e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:54.706303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fb89c4b9bc04bde91fb1bae662d5489', 'request_id': 'req-0b940211-743b-4e54-a50f-25ccadc4cd2e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:54.766308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fb89c4b9bc04bde91fb1bae662d5489', 'request_id': 'req-0b940211-743b-4e54-a50f-25ccadc4cd2e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:54.778181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] The token cache is being invalidate because project 5fb89c4b9bc04bde91fb1bae662d5489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:14:54.778472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:54.779103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:54.783693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1160/2314] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:14:54 2022] DELETE /identity/v3/projects/5fb89c4b9bc04bde91fb1bae662d5489 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:54.784821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8037b-b00d-4b52-bbf8-9310a0ab70c4 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:54.794631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8037b-b00d-4b52-bbf8-9310a0ab70c4 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:54.832197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:54.859955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:54.860614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:54.860893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:54.861163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] PATH_INFO: `/v3/users/eeb7debd3ed64db381a5c41e4474630f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:54.862324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb7debd3ed64db381a5c41e4474630f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:54.864255 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:55.066804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb7debd3ed64db381a5c41e4474630f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.149266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb7debd3ed64db381a5c41e4474630f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.157468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb7debd3ed64db381a5c41e4474630f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.158106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb7debd3ed64db381a5c41e4474630f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.158713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-47cb5e84-8feb-47fc-9bb4-a19045da08d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb7debd3ed64db381a5c41e4474630f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.186874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1155/2315] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:14:54 2022] DELETE /identity/v3/users/eeb7debd3ed64db381a5c41e4474630f => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:55.573214 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:55.574701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:55.588833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b940211-743b-4e54-a50f-25ccadc4cd2e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:55.645132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:55.645763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:55.646045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:55.646316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] PATH_INFO: `/v3/projects/68069624eec94d199e1a91d9401b63f2` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:55.647436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=68069624eec94d199e1a91d9401b63f2)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:55.649590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:55.731672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68069624eec94d199e1a91d9401b63f2', 'request_id': 'req-09099273-c22a-4863-85d1-c40911c4c7fb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.737278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68069624eec94d199e1a91d9401b63f2', 'request_id': 'req-09099273-c22a-4863-85d1-c40911c4c7fb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.777376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68069624eec94d199e1a91d9401b63f2', 'request_id': 'req-09099273-c22a-4863-85d1-c40911c4c7fb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.789233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] The token cache is being invalidate because project 68069624eec94d199e1a91d9401b63f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:14:55.789519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.790155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-09099273-c22a-4863-85d1-c40911c4c7fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:55.791644 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1161/2316] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:14:55 2022] DELETE /identity/v3/projects/68069624eec94d199e1a91d9401b63f2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:55.796935 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd48477-d701-4180-a62e-b1ab07907fd7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:55.797319 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd48477-d701-4180-a62e-b1ab07907fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:55.797691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd48477-d701-4180-a62e-b1ab07907fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:55.817173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0bd48477-d701-4180-a62e-b1ab07907fd7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:55.818361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0bd48477-d701-4180-a62e-b1ab07907fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:55.880020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1156/2317] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:55.887976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd8d02-1b21-4bd2-b4f9-90ff09297398 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:55.888326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd8d02-1b21-4bd2-b4f9-90ff09297398 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:55.888651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd8d02-1b21-4bd2-b4f9-90ff09297398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:55.912224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d9dd8d02-1b21-4bd2-b4f9-90ff09297398 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:55.913743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d9dd8d02-1b21-4bd2-b4f9-90ff09297398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:55.923626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1162/2318] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:55.928534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd48477-d701-4180-a62e-b1ab07907fd7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:55.940460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd48477-d701-4180-a62e-b1ab07907fd7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:55.945948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa90223-a3ab-4b72-ae3d-fe7d52bf677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:55.948385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa90223-a3ab-4b72-ae3d-fe7d52bf677e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:55.948385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa90223-a3ab-4b72-ae3d-fe7d52bf677e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:55.948385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa90223-a3ab-4b72-ae3d-fe7d52bf677e admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:55.948385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa90223-a3ab-4b72-ae3d-fe7d52bf677e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:14:55.948385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa90223-a3ab-4b72-ae3d-fe7d52bf677e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:55.950216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa90223-a3ab-4b72-ae3d-fe7d52bf677e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:55.963658 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1157/2319] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:14:55.968222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd8d02-1b21-4bd2-b4f9-90ff09297398 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:55.972912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:55.978371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd8d02-1b21-4bd2-b4f9-90ff09297398 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:55.983307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d042f3-5e70-4411-973e-d5b07d2ce4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:55.983835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d042f3-5e70-4411-973e-d5b07d2ce4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:55.984109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d042f3-5e70-4411-973e-d5b07d2ce4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:55.984372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d042f3-5e70-4411-973e-d5b07d2ce4d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:55.985029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d042f3-5e70-4411-973e-d5b07d2ce4d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:55.986957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d042f3-5e70-4411-973e-d5b07d2ce4d1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:56.087362 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1163/2320] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:14:55 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:14:56.091968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa90223-a3ab-4b72-ae3d-fe7d52bf677e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:56.102308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa90223-a3ab-4b72-ae3d-fe7d52bf677e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:56.137231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:56.161147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e721c2-f19d-45d6-b988-8ffbd29b680c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:56.161359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e721c2-f19d-45d6-b988-8ffbd29b680c admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:56.161595 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e721c2-f19d-45d6-b988-8ffbd29b680c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:56.161860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e721c2-f19d-45d6-b988-8ffbd29b680c admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:56.162487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e721c2-f19d-45d6-b988-8ffbd29b680c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:56.164449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e721c2-f19d-45d6-b988-8ffbd29b680c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:56.250353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1158/2321] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:14:56 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:14:56.252106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d042f3-5e70-4411-973e-d5b07d2ce4d1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:56.263508 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d042f3-5e70-4411-973e-d5b07d2ce4d1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:56.270712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f571c8-3bb2-4b50-9007-a1718bc26d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:56.271355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f571c8-3bb2-4b50-9007-a1718bc26d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:56.271617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f571c8-3bb2-4b50-9007-a1718bc26d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:56.271885 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f571c8-3bb2-4b50-9007-a1718bc26d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:56.272556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f571c8-3bb2-4b50-9007-a1718bc26d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:56.274303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f571c8-3bb2-4b50-9007-a1718bc26d39 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:56.287037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1164/2322] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:14:56.297649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e721c2-f19d-45d6-b988-8ffbd29b680c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:56.308835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e721c2-f19d-45d6-b988-8ffbd29b680c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:56.314286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534dc91-4b38-4895-8513-da503f6f4808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:56.314889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534dc91-4b38-4895-8513-da503f6f4808 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:56.315168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534dc91-4b38-4895-8513-da503f6f4808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:56.315439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534dc91-4b38-4895-8513-da503f6f4808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:56.316083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534dc91-4b38-4895-8513-da503f6f4808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:56.318159 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534dc91-4b38-4895-8513-da503f6f4808 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:56.329209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1159/2323] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:14:56.333316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f571c8-3bb2-4b50-9007-a1718bc26d39 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:56.344455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f571c8-3bb2-4b50-9007-a1718bc26d39 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:56.349242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16329c04-e45b-401a-aa72-ad7313c8ebfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:56.349922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16329c04-e45b-401a-aa72-ad7313c8ebfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:56.350203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16329c04-e45b-401a-aa72-ad7313c8ebfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:56.350472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16329c04-e45b-401a-aa72-ad7313c8ebfd admin admin] PATH_INFO: `/v3/projects/b834ddad792049ab9110b840ab07a7a6/users/128f9f5b19784bd8afb313610b93673d/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:56.367676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16329c04-e45b-401a-aa72-ad7313c8ebfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b834ddad792049ab9110b840ab07a7a6, user_id=128f9f5b19784bd8afb313610b93673d, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:56.370427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16329c04-e45b-401a-aa72-ad7313c8ebfd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:56.442596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1165/2324] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:14:56 2022] PUT /identity/v3/projects/b834ddad792049ab9110b840ab07a7a6/users/128f9f5b19784bd8afb313610b93673d/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:56.454712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad6a89-1319-4a94-9531-9356c5f43de4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:56.454996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad6a89-1319-4a94-9531-9356c5f43de4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:56.455267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad6a89-1319-4a94-9531-9356c5f43de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:56.486191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6bad6a89-1319-4a94-9531-9356c5f43de4 None None] MFA Rules not processed for user `128f9f5b19784bd8afb313610b93673d`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:56.487428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6bad6a89-1319-4a94-9531-9356c5f43de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:56.556379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1160/2325] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:56.568034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16329c04-e45b-401a-aa72-ad7313c8ebfd admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:56.569756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-16329c04-e45b-401a-aa72-ad7313c8ebfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:56.627762 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16329c04-e45b-401a-aa72-ad7313c8ebfd admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:56.631607 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d202964-5464-417d-a88f-f88f8d3403a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:56.632210 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d202964-5464-417d-a88f-f88f8d3403a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:56.632489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d202964-5464-417d-a88f-f88f8d3403a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:56.632785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d202964-5464-417d-a88f-f88f8d3403a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:56.641091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d202964-5464-417d-a88f-f88f8d3403a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:56.642519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d202964-5464-417d-a88f-f88f8d3403a4 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:56.651175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1166/2326] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:56.674421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad6a89-1319-4a94-9531-9356c5f43de4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:56.738052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad6a89-1319-4a94-9531-9356c5f43de4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:56.742773 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:56.743391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:56.743671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:56.743938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] PATH_INFO: `/v3/users/128f9f5b19784bd8afb313610b93673d` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:56.745085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=128f9f5b19784bd8afb313610b93673d)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:56.747007 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:56.934165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128f9f5b19784bd8afb313610b93673d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:56.935816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:57.006763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128f9f5b19784bd8afb313610b93673d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:57.014016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128f9f5b19784bd8afb313610b93673d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:57.018791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128f9f5b19784bd8afb313610b93673d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:57.018791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128f9f5b19784bd8afb313610b93673d'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:57.032911 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1161/2327] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:14:56 2022] DELETE /identity/v3/users/128f9f5b19784bd8afb313610b93673d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:57.037483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa53380-ed74-4cad-8795-0ecbaff4af57 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:57.037788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa53380-ed74-4cad-8795-0ecbaff4af57 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:57.038062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa53380-ed74-4cad-8795-0ecbaff4af57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:57.057133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-baa53380-ed74-4cad-8795-0ecbaff4af57 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:57.058315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-baa53380-ed74-4cad-8795-0ecbaff4af57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:57.116940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1167/2328] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:57.127562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:57.129040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:57.186227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d7661-104f-4bf5-9758-73903c39a813 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:57.190058 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024e8f7-d24e-4e35-a972-8c20230836e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:57.190653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024e8f7-d24e-4e35-a972-8c20230836e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:57.190928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024e8f7-d24e-4e35-a972-8c20230836e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:57.191221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024e8f7-d24e-4e35-a972-8c20230836e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:57.200412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024e8f7-d24e-4e35-a972-8c20230836e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:57.202718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024e8f7-d24e-4e35-a972-8c20230836e2 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:57.212359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1162/2329] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:14:57.659405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa53380-ed74-4cad-8795-0ecbaff4af57 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:57.663183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-baa53380-ed74-4cad-8795-0ecbaff4af57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:57.673286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa53380-ed74-4cad-8795-0ecbaff4af57 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:57.678054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:57.678683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:57.678960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:57.679230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] PATH_INFO: `/v3/projects/b834ddad792049ab9110b840ab07a7a6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:57.680354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b834ddad792049ab9110b840ab07a7a6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:57.682310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:57.798547 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:57.802122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b834ddad792049ab9110b840ab07a7a6', 'request_id': 'req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:57.808023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b834ddad792049ab9110b840ab07a7a6', 'request_id': 'req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:57.868154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b834ddad792049ab9110b840ab07a7a6', 'request_id': 'req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:57.880944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] The token cache is being invalidate because project b834ddad792049ab9110b840ab07a7a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:14:57.881229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:57.881867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-822a8d92-a5ee-43cb-9e61-1bfd729aa94f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:14:57.883448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1168/2330] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:14:57 2022] DELETE /identity/v3/projects/b834ddad792049ab9110b840ab07a7a6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:57.888418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32876e11-a1b9-4200-a685-39d62a2717de None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:57.892382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32876e11-a1b9-4200-a685-39d62a2717de None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:57.892382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32876e11-a1b9-4200-a685-39d62a2717de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:57.908261 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-32876e11-a1b9-4200-a685-39d62a2717de None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:57.909494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-32876e11-a1b9-4200-a685-39d62a2717de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:57.970312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1163/2331] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:57.977990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adedd93-fa45-4af9-8630-5b35a7192e27 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:57.978278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adedd93-fa45-4af9-8630-5b35a7192e27 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:57.978551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adedd93-fa45-4af9-8630-5b35a7192e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:57.999625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4adedd93-fa45-4af9-8630-5b35a7192e27 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:58.001130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4adedd93-fa45-4af9-8630-5b35a7192e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:58.009186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1169/2332] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:58.013612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32876e11-a1b9-4200-a685-39d62a2717de None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:58.023340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32876e11-a1b9-4200-a685-39d62a2717de None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:58.028128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42affe-5540-4f5e-9cf2-4c643d76c74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:58.028768 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42affe-5540-4f5e-9cf2-4c643d76c74d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:58.029045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42affe-5540-4f5e-9cf2-4c643d76c74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:58.029312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42affe-5540-4f5e-9cf2-4c643d76c74d admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:58.029951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42affe-5540-4f5e-9cf2-4c643d76c74d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:14:58.030246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42affe-5540-4f5e-9cf2-4c643d76c74d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:58.031964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42affe-5540-4f5e-9cf2-4c643d76c74d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:58.043400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1164/2333] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:14:58.048114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adedd93-fa45-4af9-8630-5b35a7192e27 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:58.058445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adedd93-fa45-4af9-8630-5b35a7192e27 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:58.063680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70a4c7-e006-48c6-bdc0-fa353ae5621f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:58.064264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70a4c7-e006-48c6-bdc0-fa353ae5621f admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:58.064571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70a4c7-e006-48c6-bdc0-fa353ae5621f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:58.064839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70a4c7-e006-48c6-bdc0-fa353ae5621f admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:58.065461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70a4c7-e006-48c6-bdc0-fa353ae5621f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:58.067345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70a4c7-e006-48c6-bdc0-fa353ae5621f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:58.179364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1170/2334] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:14:58 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:14:58.183802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42affe-5540-4f5e-9cf2-4c643d76c74d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:58.199050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42affe-5540-4f5e-9cf2-4c643d76c74d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:58.250026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:58.256820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b79b-d381-4aed-94dc-a8d63f2fe4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:58.257495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b79b-d381-4aed-94dc-a8d63f2fe4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:58.257843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b79b-d381-4aed-94dc-a8d63f2fe4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:58.258212 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b79b-d381-4aed-94dc-a8d63f2fe4cb admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:58.258917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b79b-d381-4aed-94dc-a8d63f2fe4cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:58.260939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b79b-d381-4aed-94dc-a8d63f2fe4cb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:58.398988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1165/2335] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:14:58 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:14:58.405207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70a4c7-e006-48c6-bdc0-fa353ae5621f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:58.413197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70a4c7-e006-48c6-bdc0-fa353ae5621f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:58.417932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f0f02-3223-4abc-8af7-10b0fd33cbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:58.418585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0f02-3223-4abc-8af7-10b0fd33cbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:58.418859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0f02-3223-4abc-8af7-10b0fd33cbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:58.419129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0f02-3223-4abc-8af7-10b0fd33cbc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:58.419760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f0f02-3223-4abc-8af7-10b0fd33cbc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:58.422053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f0f02-3223-4abc-8af7-10b0fd33cbc8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:58.432916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1171/2336] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:14:58.437010 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b79b-d381-4aed-94dc-a8d63f2fe4cb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:58.446796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b79b-d381-4aed-94dc-a8d63f2fe4cb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:58.451420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97350be-1d34-47f2-8dd1-ae5d4b6fb353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:58.456266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97350be-1d34-47f2-8dd1-ae5d4b6fb353 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:58.456266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97350be-1d34-47f2-8dd1-ae5d4b6fb353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:58.456266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97350be-1d34-47f2-8dd1-ae5d4b6fb353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:58.456266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97350be-1d34-47f2-8dd1-ae5d4b6fb353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:58.456266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97350be-1d34-47f2-8dd1-ae5d4b6fb353 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:58.470033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1166/2337] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:14:58.472490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f0f02-3223-4abc-8af7-10b0fd33cbc8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:58.483503 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f0f02-3223-4abc-8af7-10b0fd33cbc8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:58.488347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde370-2c9c-4868-842f-e8ed1cf43a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:58.489135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde370-2c9c-4868-842f-e8ed1cf43a2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:58.489413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde370-2c9c-4868-842f-e8ed1cf43a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:58.489680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde370-2c9c-4868-842f-e8ed1cf43a2c admin admin] PATH_INFO: `/v3/projects/a9703306bcfb4df98f36561a65b71d7c/users/f322de3bf60b4416bf0b958922c2cc54/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:58.506218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fde370-2c9c-4868-842f-e8ed1cf43a2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9703306bcfb4df98f36561a65b71d7c, user_id=f322de3bf60b4416bf0b958922c2cc54, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:58.509745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fde370-2c9c-4868-842f-e8ed1cf43a2c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:58.590765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1172/2338] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:14:58 2022] PUT /identity/v3/projects/a9703306bcfb4df98f36561a65b71d7c/users/f322de3bf60b4416bf0b958922c2cc54/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:14:58.607972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f748-5776-4861-8342-eda57097b9f0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:58.608345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f748-5776-4861-8342-eda57097b9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:58.608714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f748-5776-4861-8342-eda57097b9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:58.640651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5201f748-5776-4861-8342-eda57097b9f0 None None] MFA Rules not processed for user `f322de3bf60b4416bf0b958922c2cc54`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:58.641917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5201f748-5776-4861-8342-eda57097b9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:58.725540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1167/2339] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:14:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:14:58.734052 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde370-2c9c-4868-842f-e8ed1cf43a2c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:58.735870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b0fde370-2c9c-4868-842f-e8ed1cf43a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:58.799283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde370-2c9c-4868-842f-e8ed1cf43a2c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:58.800425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aef8a-47b0-4929-8fdb-fbe57da9590a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:58.800425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aef8a-47b0-4929-8fdb-fbe57da9590a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:58.800838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aef8a-47b0-4929-8fdb-fbe57da9590a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:58.805694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efbc32-3f21-4190-8632-f9fd61f815e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:58.806317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efbc32-3f21-4190-8632-f9fd61f815e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:58.806600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efbc32-3f21-4190-8632-f9fd61f815e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:58.806874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efbc32-3f21-4190-8632-f9fd61f815e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:58.817226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efbc32-3f21-4190-8632-f9fd61f815e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:58.818684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efbc32-3f21-4190-8632-f9fd61f815e1 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:58.821368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fd9aef8a-47b0-4929-8fdb-fbe57da9590a None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:14:58.822649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fd9aef8a-47b0-4929-8fdb-fbe57da9590a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:58.829299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1173/2340] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:14:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:14:58.893011 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1168/2341] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:14:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:14:59.033048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efbc32-3f21-4190-8632-f9fd61f815e1 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:14:59.034439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d3efbc32-3f21-4190-8632-f9fd61f815e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:14:59.103992 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efbc32-3f21-4190-8632-f9fd61f815e1 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:14:59.109170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01091d4-2027-4305-9d88-2b09b9ac2c62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:14:59.109763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01091d4-2027-4305-9d88-2b09b9ac2c62 service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:14:59.110062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01091d4-2027-4305-9d88-2b09b9ac2c62 service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:14:59.110328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01091d4-2027-4305-9d88-2b09b9ac2c62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:14:59.125784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01091d4-2027-4305-9d88-2b09b9ac2c62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:14:59.125784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01091d4-2027-4305-9d88-2b09b9ac2c62 service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:14:59.127610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:14:59.133690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1174/2342] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:14:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:15:08.883631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:09.332421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9aef8a-47b0-4929-8fdb-fbe57da9590a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:09.426619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9aef8a-47b0-4929-8fdb-fbe57da9590a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:09.431468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:09.432095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:09.432415 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:09.432716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] PATH_INFO: `/v3/users/f322de3bf60b4416bf0b958922c2cc54` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:09.433836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f322de3bf60b4416bf0b958922c2cc54)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:09.438435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:09.632597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f322de3bf60b4416bf0b958922c2cc54'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:09.704657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f322de3bf60b4416bf0b958922c2cc54'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:09.713268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f322de3bf60b4416bf0b958922c2cc54'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:09.713916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f322de3bf60b4416bf0b958922c2cc54'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:09.714679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f322de3bf60b4416bf0b958922c2cc54'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:09.735282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1169/2343] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:15:09 2022] DELETE /identity/v3/users/f322de3bf60b4416bf0b958922c2cc54 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:09.739805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de680c86-abe1-40d9-9273-e8a6b0607b83 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:09.740287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de680c86-abe1-40d9-9273-e8a6b0607b83 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:09.740399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de680c86-abe1-40d9-9273-e8a6b0607b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:09.762335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-de680c86-abe1-40d9-9273-e8a6b0607b83 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:09.763581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-de680c86-abe1-40d9-9273-e8a6b0607b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:09.833476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1175/2344] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:09.843398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:09.844840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:09.904577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23330411-f3a8-4b42-910c-7a3d679d1a26 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:09.908074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f347ef-edca-47bf-91bd-62f9b5fdda9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:09.908686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f347ef-edca-47bf-91bd-62f9b5fdda9a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:09.908958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f347ef-edca-47bf-91bd-62f9b5fdda9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:09.909220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f347ef-edca-47bf-91bd-62f9b5fdda9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:09.917976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f347ef-edca-47bf-91bd-62f9b5fdda9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:09.919483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f347ef-edca-47bf-91bd-62f9b5fdda9a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:09.929352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1170/2345] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:10.322079 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de680c86-abe1-40d9-9273-e8a6b0607b83 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:10.323638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-de680c86-abe1-40d9-9273-e8a6b0607b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:10.341614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de680c86-abe1-40d9-9273-e8a6b0607b83 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:10.346977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:10.347970 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:10.348312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:10.348785 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] PATH_INFO: `/v3/projects/a9703306bcfb4df98f36561a65b71d7c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:10.350051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9703306bcfb4df98f36561a65b71d7c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:10.352057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:10.483902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9703306bcfb4df98f36561a65b71d7c', 'request_id': 'req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:10.490238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9703306bcfb4df98f36561a65b71d7c', 'request_id': 'req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:10.555274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9703306bcfb4df98f36561a65b71d7c', 'request_id': 'req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:10.568617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] The token cache is being invalidate because project a9703306bcfb4df98f36561a65b71d7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:15:10.568904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:10.569555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b19fa0ea-54ea-4f0d-b611-5ed9e5589be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:10.571049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1176/2346] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:15:10 2022] DELETE /identity/v3/projects/a9703306bcfb4df98f36561a65b71d7c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:10.576168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c05b50-a2b9-4544-a270-299f6e625f08 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:10.576516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c05b50-a2b9-4544-a270-299f6e625f08 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:10.576786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c05b50-a2b9-4544-a270-299f6e625f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:10.597223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-57c05b50-a2b9-4544-a270-299f6e625f08 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:10.598412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-57c05b50-a2b9-4544-a270-299f6e625f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:10.615364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:10.657012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1171/2347] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:10.665805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c9794-10f5-45d8-95f4-e0a492f8a7c2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:10.666082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c9794-10f5-45d8-95f4-e0a492f8a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:10.666373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c9794-10f5-45d8-95f4-e0a492f8a7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:10.687507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fa2c9794-10f5-45d8-95f4-e0a492f8a7c2 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:10.688739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fa2c9794-10f5-45d8-95f4-e0a492f8a7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:10.696991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1177/2348] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:10.701153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c05b50-a2b9-4544-a270-299f6e625f08 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:10.713798 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c05b50-a2b9-4544-a270-299f6e625f08 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:10.716667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e8afa-adb2-41ec-a585-c03a5c57f0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:10.717359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e8afa-adb2-41ec-a585-c03a5c57f0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:10.717546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e8afa-adb2-41ec-a585-c03a5c57f0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:10.717808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e8afa-adb2-41ec-a585-c03a5c57f0ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:10.718557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e8afa-adb2-41ec-a585-c03a5c57f0ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:15:10.718765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e8afa-adb2-41ec-a585-c03a5c57f0ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:10.720384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e8afa-adb2-41ec-a585-c03a5c57f0ad admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:10.732193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1172/2349] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:15:10.737220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c9794-10f5-45d8-95f4-e0a492f8a7c2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:10.746388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c9794-10f5-45d8-95f4-e0a492f8a7c2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:10.751652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb036326-8aa3-4556-b8cb-8d4456cfc92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:10.752237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb036326-8aa3-4556-b8cb-8d4456cfc92b admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:10.752552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb036326-8aa3-4556-b8cb-8d4456cfc92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:10.752802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb036326-8aa3-4556-b8cb-8d4456cfc92b admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:10.753427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb036326-8aa3-4556-b8cb-8d4456cfc92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:10.755304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb036326-8aa3-4556-b8cb-8d4456cfc92b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:10.873513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1178/2350] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:15:10 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:10.878066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e8afa-adb2-41ec-a585-c03a5c57f0ad admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:10.888141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e8afa-adb2-41ec-a585-c03a5c57f0ad admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:10.943025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3939b-5bab-4745-8de9-107e84326718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:10.943660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3939b-5bab-4745-8de9-107e84326718 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:10.943876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3939b-5bab-4745-8de9-107e84326718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:10.944143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3939b-5bab-4745-8de9-107e84326718 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:10.944782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3939b-5bab-4745-8de9-107e84326718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:10.946685 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3939b-5bab-4745-8de9-107e84326718 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:11.018647 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1173/2351] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:15:10 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:11.022678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb036326-8aa3-4556-b8cb-8d4456cfc92b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:11.033937 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb036326-8aa3-4556-b8cb-8d4456cfc92b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:11.038853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cebca-e41a-4b5b-8163-007e24539dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:11.039478 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cebca-e41a-4b5b-8163-007e24539dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.039757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cebca-e41a-4b5b-8163-007e24539dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.040028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cebca-e41a-4b5b-8163-007e24539dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.040699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cebca-e41a-4b5b-8163-007e24539dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:11.042306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cebca-e41a-4b5b-8163-007e24539dd6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:11.053347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1179/2352] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:11.057549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3939b-5bab-4745-8de9-107e84326718 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:11.068057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3939b-5bab-4745-8de9-107e84326718 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:11.072853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86f4df-3907-4df3-a000-29d115d6886e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:11.073443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f4df-3907-4df3-a000-29d115d6886e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.073745 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f4df-3907-4df3-a000-29d115d6886e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.074017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86f4df-3907-4df3-a000-29d115d6886e admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.074656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86f4df-3907-4df3-a000-29d115d6886e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:11.076319 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86f4df-3907-4df3-a000-29d115d6886e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:11.090977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1174/2353] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:15:11.092162 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cebca-e41a-4b5b-8163-007e24539dd6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:11.102796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cebca-e41a-4b5b-8163-007e24539dd6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:11.107380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9199149-6bd1-4675-b384-fc565091cc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:11.108012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9199149-6bd1-4675-b384-fc565091cc0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.110134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9199149-6bd1-4675-b384-fc565091cc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.110134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9199149-6bd1-4675-b384-fc565091cc0d admin admin] PATH_INFO: `/v3/projects/48d10cd664254ab1bb9485d424a14bc6/users/ebfa3da736d5461c999d8e67867c2a98/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.124411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9199149-6bd1-4675-b384-fc565091cc0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d10cd664254ab1bb9485d424a14bc6, user_id=ebfa3da736d5461c999d8e67867c2a98, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:11.127092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9199149-6bd1-4675-b384-fc565091cc0d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:11.195356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1180/2354] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:15:11 2022] PUT /identity/v3/projects/48d10cd664254ab1bb9485d424a14bc6/users/ebfa3da736d5461c999d8e67867c2a98/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:11.206599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7f781-e794-48e1-a853-33327ab2d5a1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.206883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7f781-e794-48e1-a853-33327ab2d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.207156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7f781-e794-48e1-a853-33327ab2d5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.235912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e5d7f781-e794-48e1-a853-33327ab2d5a1 None None] MFA Rules not processed for user `ebfa3da736d5461c999d8e67867c2a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:11.237181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e5d7f781-e794-48e1-a853-33327ab2d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:11.306125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1175/2355] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:11.310616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9199149-6bd1-4675-b384-fc565091cc0d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:11.332107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:11.374287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9199149-6bd1-4675-b384-fc565091cc0d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:11.379158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d3896-7295-4f27-aad5-8462f28c1ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:11.379757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d3896-7295-4f27-aad5-8462f28c1ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.380030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d3896-7295-4f27-aad5-8462f28c1ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.380295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d3896-7295-4f27-aad5-8462f28c1ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.380983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d3896-7295-4f27-aad5-8462f28c1ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:11.382946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d3896-7295-4f27-aad5-8462f28c1ee2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:11.477862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1181/2356] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:15:11 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:11.481645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7f781-e794-48e1-a853-33327ab2d5a1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:11.492303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7f781-e794-48e1-a853-33327ab2d5a1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:11.504281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:11.549847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042061c9-302a-4c79-beb2-58aea7da42a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:11.550751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042061c9-302a-4c79-beb2-58aea7da42a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.551025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042061c9-302a-4c79-beb2-58aea7da42a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.551290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042061c9-302a-4c79-beb2-58aea7da42a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.551915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042061c9-302a-4c79-beb2-58aea7da42a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:11.553877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042061c9-302a-4c79-beb2-58aea7da42a6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:11.631220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1176/2357] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:15:11 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:11.635964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d3896-7295-4f27-aad5-8462f28c1ee2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:11.646871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d3896-7295-4f27-aad5-8462f28c1ee2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:11.651497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70e3a2-f79b-4e95-b134-d491967e6749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:11.652087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70e3a2-f79b-4e95-b134-d491967e6749 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.652363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70e3a2-f79b-4e95-b134-d491967e6749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.652662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70e3a2-f79b-4e95-b134-d491967e6749 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.653295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70e3a2-f79b-4e95-b134-d491967e6749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:11.654875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70e3a2-f79b-4e95-b134-d491967e6749 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:11.666081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1182/2358] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:11.669942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042061c9-302a-4c79-beb2-58aea7da42a6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:11.679912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042061c9-302a-4c79-beb2-58aea7da42a6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:11.684724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2e60a-0a8b-437c-99b4-c08d541e0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:11.685308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2e60a-0a8b-437c-99b4-c08d541e0759 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.685586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2e60a-0a8b-437c-99b4-c08d541e0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.685853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2e60a-0a8b-437c-99b4-c08d541e0759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.686486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2e60a-0a8b-437c-99b4-c08d541e0759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:11.688577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2e60a-0a8b-437c-99b4-c08d541e0759 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:11.698375 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1177/2359] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:11.702966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70e3a2-f79b-4e95-b134-d491967e6749 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:11.713152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70e3a2-f79b-4e95-b134-d491967e6749 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:11.717700 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6ccd4-3c5f-4627-b212-8c7086726c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:11.718335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6ccd4-3c5f-4627-b212-8c7086726c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.718611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6ccd4-3c5f-4627-b212-8c7086726c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.718880 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6ccd4-3c5f-4627-b212-8c7086726c22 admin admin] PATH_INFO: `/v3/projects/bc91bf2f6d1f40d68ae9c5f729f35e52/users/a83c841f0b204e96a1dc60d36b47029e/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.733062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6ccd4-3c5f-4627-b212-8c7086726c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc91bf2f6d1f40d68ae9c5f729f35e52, user_id=a83c841f0b204e96a1dc60d36b47029e, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:11.735717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6ccd4-3c5f-4627-b212-8c7086726c22 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:11.806924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1183/2360] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:15:11 2022] PUT /identity/v3/projects/bc91bf2f6d1f40d68ae9c5f729f35e52/users/a83c841f0b204e96a1dc60d36b47029e/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:11.942545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2219306-ec9a-48fb-a959-9951d9f5c81a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.942545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2219306-ec9a-48fb-a959-9951d9f5c81a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.942545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2219306-ec9a-48fb-a959-9951d9f5c81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.942545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e2219306-ec9a-48fb-a959-9951d9f5c81a None None] MFA Rules not processed for user `a83c841f0b204e96a1dc60d36b47029e`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:11.942545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e2219306-ec9a-48fb-a959-9951d9f5c81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:11.942545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1178/2361] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:11.944322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6ccd4-3c5f-4627-b212-8c7086726c22 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:11.990284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6ccd4-3c5f-4627-b212-8c7086726c22 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:11.994999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dfc4e-e070-4a53-9318-56492bc8b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:11.995593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dfc4e-e070-4a53-9318-56492bc8b5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:11.995875 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dfc4e-e070-4a53-9318-56492bc8b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:11.996145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dfc4e-e070-4a53-9318-56492bc8b5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:11.996805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6dfc4e-e070-4a53-9318-56492bc8b5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:11.998736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6dfc4e-e070-4a53-9318-56492bc8b5ec admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:12.093839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1184/2362] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:15:11 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:12.097749 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2219306-ec9a-48fb-a959-9951d9f5c81a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:12.109195 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2219306-ec9a-48fb-a959-9951d9f5c81a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:12.165816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e190d62-7cdc-4bf9-acc9-e72635325802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:12.166412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e190d62-7cdc-4bf9-acc9-e72635325802 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:12.166686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e190d62-7cdc-4bf9-acc9-e72635325802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:12.166953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e190d62-7cdc-4bf9-acc9-e72635325802 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:12.167576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e190d62-7cdc-4bf9-acc9-e72635325802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:12.169538 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e190d62-7cdc-4bf9-acc9-e72635325802 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:12.263369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1179/2363] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:15:12 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:12.267355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dfc4e-e070-4a53-9318-56492bc8b5ec admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:12.277900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dfc4e-e070-4a53-9318-56492bc8b5ec admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:12.282678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033894f-a2bc-4c5c-8146-1af4dcf9a753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:12.283311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033894f-a2bc-4c5c-8146-1af4dcf9a753 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:12.283589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033894f-a2bc-4c5c-8146-1af4dcf9a753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:12.283855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033894f-a2bc-4c5c-8146-1af4dcf9a753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:12.284545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033894f-a2bc-4c5c-8146-1af4dcf9a753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:12.286136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033894f-a2bc-4c5c-8146-1af4dcf9a753 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:12.297020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1185/2364] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:12.301090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e190d62-7cdc-4bf9-acc9-e72635325802 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:12.311071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e190d62-7cdc-4bf9-acc9-e72635325802 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:12.316233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77602a73-5fc8-434e-8b81-772e6a163665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:12.316900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77602a73-5fc8-434e-8b81-772e6a163665 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:12.317175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77602a73-5fc8-434e-8b81-772e6a163665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:12.317442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77602a73-5fc8-434e-8b81-772e6a163665 admin admin] PATH_INFO: `/v3/projects/63091549e43d4dc7a4475a0f0e1dbdf2/users/b8596df9df164ef398535ec30c8e002b/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:12.330420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:12.332959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77602a73-5fc8-434e-8b81-772e6a163665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63091549e43d4dc7a4475a0f0e1dbdf2, user_id=b8596df9df164ef398535ec30c8e002b, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:12.335599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77602a73-5fc8-434e-8b81-772e6a163665 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:12.439238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1180/2365] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:15:12 2022] PUT /identity/v3/projects/63091549e43d4dc7a4475a0f0e1dbdf2/users/b8596df9df164ef398535ec30c8e002b/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:15:12.439500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033894f-a2bc-4c5c-8146-1af4dcf9a753 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:12.439500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033894f-a2bc-4c5c-8146-1af4dcf9a753 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:12.472763 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:12.487621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fce36-ddc2-47a8-96fc-45b080ec48e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:12.489306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fce36-ddc2-47a8-96fc-45b080ec48e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:12.489306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fce36-ddc2-47a8-96fc-45b080ec48e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:12.489306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fce36-ddc2-47a8-96fc-45b080ec48e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:12.489613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fce36-ddc2-47a8-96fc-45b080ec48e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:12.490968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fce36-ddc2-47a8-96fc-45b080ec48e1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:12.500328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1186/2366] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:12.504417 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77602a73-5fc8-434e-8b81-772e6a163665 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:12.514530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77602a73-5fc8-434e-8b81-772e6a163665 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:12.519206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950eed2-594e-445d-ad20-321ef7ab8cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:12.519838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950eed2-594e-445d-ad20-321ef7ab8cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:12.520112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950eed2-594e-445d-ad20-321ef7ab8cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:12.520379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950eed2-594e-445d-ad20-321ef7ab8cb1 admin admin] PATH_INFO: `/v3/projects/63091549e43d4dc7a4475a0f0e1dbdf2/users/b8596df9df164ef398535ec30c8e002b/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:12.522306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950eed2-594e-445d-ad20-321ef7ab8cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63091549e43d4dc7a4475a0f0e1dbdf2, user_id=b8596df9df164ef398535ec30c8e002b, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:12.524955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950eed2-594e-445d-ad20-321ef7ab8cb1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:12.535791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1181/2367] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:15:12 2022] PUT /identity/v3/projects/63091549e43d4dc7a4475a0f0e1dbdf2/users/b8596df9df164ef398535ec30c8e002b/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:12.555609 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba524e8-d3a6-4695-bfeb-ed1f4762d74a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:12.555890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba524e8-d3a6-4695-bfeb-ed1f4762d74a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:12.556160 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba524e8-d3a6-4695-bfeb-ed1f4762d74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:12.585071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8ba524e8-d3a6-4695-bfeb-ed1f4762d74a None None] MFA Rules not processed for user `b8596df9df164ef398535ec30c8e002b`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:12.586272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8ba524e8-d3a6-4695-bfeb-ed1f4762d74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:12.653598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1187/2368] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:12.665152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950eed2-594e-445d-ad20-321ef7ab8cb1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:12.666839 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e950eed2-594e-445d-ad20-321ef7ab8cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:12.726784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950eed2-594e-445d-ad20-321ef7ab8cb1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:12.730435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1500cd0-3032-48e0-8383-735d34dec3bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:12.731031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500cd0-3032-48e0-8383-735d34dec3bc service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:12.731306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500cd0-3032-48e0-8383-735d34dec3bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:12.731574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500cd0-3032-48e0-8383-735d34dec3bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:12.785973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1500cd0-3032-48e0-8383-735d34dec3bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:12.787400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1500cd0-3032-48e0-8383-735d34dec3bc service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:12.797205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1182/2369] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:14.779550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba524e8-d3a6-4695-bfeb-ed1f4762d74a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:14.781031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8ba524e8-d3a6-4695-bfeb-ed1f4762d74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:14.793276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba524e8-d3a6-4695-bfeb-ed1f4762d74a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:14.797037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccff02-4c53-41bb-b0fe-c82539c10d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:14.797642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccff02-4c53-41bb-b0fe-c82539c10d35 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:14.797918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccff02-4c53-41bb-b0fe-c82539c10d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:14.798213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccff02-4c53-41bb-b0fe-c82539c10d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:14.856201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccff02-4c53-41bb-b0fe-c82539c10d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:14.857671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccff02-4c53-41bb-b0fe-c82539c10d35 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:14.866819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1188/2370] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:16.202101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1500cd0-3032-48e0-8383-735d34dec3bc service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:16.212376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1500cd0-3032-48e0-8383-735d34dec3bc service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:16.216686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396867c9-ed2d-4cc7-b9d7-949acc3e44ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:16.217554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396867c9-ed2d-4cc7-b9d7-949acc3e44ae service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:16.217837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396867c9-ed2d-4cc7-b9d7-949acc3e44ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:16.218105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396867c9-ed2d-4cc7-b9d7-949acc3e44ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:16.227408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396867c9-ed2d-4cc7-b9d7-949acc3e44ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:16.228894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396867c9-ed2d-4cc7-b9d7-949acc3e44ae service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:16.238670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1183/2371] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:19.479976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b7805-48c8-4ce0-b7f5-0458bded3123 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:19.480495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b7805-48c8-4ce0-b7f5-0458bded3123 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:19.480495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b7805-48c8-4ce0-b7f5-0458bded3123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:19.500830 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d12b7805-48c8-4ce0-b7f5-0458bded3123 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:19.502766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d12b7805-48c8-4ce0-b7f5-0458bded3123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:19.557535 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1189/2372] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:15:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:15:19.570278 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396867c9-ed2d-4cc7-b9d7-949acc3e44ae service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:19.571812 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-396867c9-ed2d-4cc7-b9d7-949acc3e44ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:19.647910 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396867c9-ed2d-4cc7-b9d7-949acc3e44ae service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:19.652813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43521a61-bb42-4f1f-b47a-6e44f4f16361 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:19.653404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43521a61-bb42-4f1f-b47a-6e44f4f16361 service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:19.653674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43521a61-bb42-4f1f-b47a-6e44f4f16361 service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:19.653942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43521a61-bb42-4f1f-b47a-6e44f4f16361 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:19.661760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43521a61-bb42-4f1f-b47a-6e44f4f16361 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:19.663396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43521a61-bb42-4f1f-b47a-6e44f4f16361 service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:19.671657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1184/2373] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:15:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:15:20.727878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b7805-48c8-4ce0-b7f5-0458bded3123 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:20.740900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:20.819149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b7805-48c8-4ce0-b7f5-0458bded3123 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:20.824183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:20.825058 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:20.825340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:20.825789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] PATH_INFO: `/v3/users/ebfa3da736d5461c999d8e67867c2a98` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:20.827731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebfa3da736d5461c999d8e67867c2a98)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:20.830199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:21.051029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebfa3da736d5461c999d8e67867c2a98'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:21.116716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebfa3da736d5461c999d8e67867c2a98'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:21.125397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebfa3da736d5461c999d8e67867c2a98'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:21.128741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebfa3da736d5461c999d8e67867c2a98'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:21.128741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebfa3da736d5461c999d8e67867c2a98'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:21.151948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1190/2374] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:15:20 2022] DELETE /identity/v3/users/ebfa3da736d5461c999d8e67867c2a98 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:21.153677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611197fe-7a2f-4c02-927f-2a5d45f9a5a0 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:21.154227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611197fe-7a2f-4c02-927f-2a5d45f9a5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:21.154227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611197fe-7a2f-4c02-927f-2a5d45f9a5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:21.173730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-611197fe-7a2f-4c02-927f-2a5d45f9a5a0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:21.174933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-611197fe-7a2f-4c02-927f-2a5d45f9a5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:21.179367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:21.241147 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1185/2375] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:21.251467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:21.252933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:21.316943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f438b-641c-4ca8-8a73-4696caebf673 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:21.322544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64526a8f-ba91-45e3-a48f-4f691dfab6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:21.323394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64526a8f-ba91-45e3-a48f-4f691dfab6ce service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:21.323888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64526a8f-ba91-45e3-a48f-4f691dfab6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:21.324157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64526a8f-ba91-45e3-a48f-4f691dfab6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:21.338756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64526a8f-ba91-45e3-a48f-4f691dfab6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:21.341737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64526a8f-ba91-45e3-a48f-4f691dfab6ce service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:21.353998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1191/2376] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:21.791909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611197fe-7a2f-4c02-927f-2a5d45f9a5a0 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:21.793296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-611197fe-7a2f-4c02-927f-2a5d45f9a5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:21.815481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611197fe-7a2f-4c02-927f-2a5d45f9a5a0 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:21.821397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:21.822651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:21.822651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:21.822864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] PATH_INFO: `/v3/projects/48d10cd664254ab1bb9485d424a14bc6` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:21.827117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d10cd664254ab1bb9485d424a14bc6)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:21.837457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:21.973275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d10cd664254ab1bb9485d424a14bc6', 'request_id': 'req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:21.979842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d10cd664254ab1bb9485d424a14bc6', 'request_id': 'req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:22.069520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d10cd664254ab1bb9485d424a14bc6', 'request_id': 'req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:22.085813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] The token cache is being invalidate because project 48d10cd664254ab1bb9485d424a14bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:15:22.086187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:22.087252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:22.089316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1186/2377] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:15:21 2022] DELETE /identity/v3/projects/48d10cd664254ab1bb9485d424a14bc6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:22.093426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64526a8f-ba91-45e3-a48f-4f691dfab6ce service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:22.103462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64526a8f-ba91-45e3-a48f-4f691dfab6ce service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:22.134289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:22.165681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:22.166310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:22.166613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:22.166882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] PATH_INFO: `/v3/users/a83c841f0b204e96a1dc60d36b47029e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:22.168006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83c841f0b204e96a1dc60d36b47029e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:22.169987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:22.321135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83c841f0b204e96a1dc60d36b47029e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:22.384033 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83c841f0b204e96a1dc60d36b47029e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:22.397420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83c841f0b204e96a1dc60d36b47029e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:22.401287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83c841f0b204e96a1dc60d36b47029e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:22.401287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83c841f0b204e96a1dc60d36b47029e'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:22.440068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1192/2378] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:15:22 2022] DELETE /identity/v3/users/a83c841f0b204e96a1dc60d36b47029e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:22.917057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:22.918549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:22.933931 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535b8ac-a6fe-42bc-b558-ef67f970e2ff admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:23.001240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:23.002253 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:23.002682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:23.003099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] PATH_INFO: `/v3/projects/bc91bf2f6d1f40d68ae9c5f729f35e52` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:23.004404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc91bf2f6d1f40d68ae9c5f729f35e52)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:23.006852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:23.119592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:23.130509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc91bf2f6d1f40d68ae9c5f729f35e52', 'request_id': 'req-4d3072e4-f351-4abb-a116-99c818e3bab9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.138301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc91bf2f6d1f40d68ae9c5f729f35e52', 'request_id': 'req-4d3072e4-f351-4abb-a116-99c818e3bab9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.220486 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc91bf2f6d1f40d68ae9c5f729f35e52', 'request_id': 'req-4d3072e4-f351-4abb-a116-99c818e3bab9'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.233902 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] The token cache is being invalidate because project bc91bf2f6d1f40d68ae9c5f729f35e52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:15:23.234175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.234884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.240253 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1187/2379] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:15:22 2022] DELETE /identity/v3/projects/bc91bf2f6d1f40d68ae9c5f729f35e52 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:23.240857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:23.242326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:23.255504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16965d-1a3c-4ddd-a798-dc95fc16e55b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:23.319297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:23.319978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:23.320294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:23.320575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] PATH_INFO: `/v3/users/b8596df9df164ef398535ec30c8e002b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:23.322277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8596df9df164ef398535ec30c8e002b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:23.324470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:23.491998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8596df9df164ef398535ec30c8e002b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.495725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:23.561095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8596df9df164ef398535ec30c8e002b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.569930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8596df9df164ef398535ec30c8e002b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.570702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8596df9df164ef398535ec30c8e002b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.571367 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-26567024-823f-4f51-8616-cd404d741fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8596df9df164ef398535ec30c8e002b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:23.591626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1193/2380] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:15:23 2022] DELETE /identity/v3/users/b8596df9df164ef398535ec30c8e002b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:24.083503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:24.085190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:24.103272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3072e4-f351-4abb-a116-99c818e3bab9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:24.177589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:24.178309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:24.178655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:24.178999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] PATH_INFO: `/v3/projects/63091549e43d4dc7a4475a0f0e1dbdf2` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:24.180322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63091549e43d4dc7a4475a0f0e1dbdf2)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:24.182493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:24.288114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63091549e43d4dc7a4475a0f0e1dbdf2', 'request_id': 'req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:24.294337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63091549e43d4dc7a4475a0f0e1dbdf2', 'request_id': 'req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:24.354880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63091549e43d4dc7a4475a0f0e1dbdf2', 'request_id': 'req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:24.368344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] The token cache is being invalidate because project 63091549e43d4dc7a4475a0f0e1dbdf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:15:24.368673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:24.369575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-eaba2ca3-6c29-40fb-b0aa-5b2cddebf332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:24.371227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1188/2381] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:15:24 2022] DELETE /identity/v3/projects/63091549e43d4dc7a4475a0f0e1dbdf2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:24.378205 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434237d-23db-45b3-a1b5-d23267c47492 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:24.378731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434237d-23db-45b3-a1b5-d23267c47492 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:24.378964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434237d-23db-45b3-a1b5-d23267c47492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:24.401151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-f434237d-23db-45b3-a1b5-d23267c47492 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:24.402384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f434237d-23db-45b3-a1b5-d23267c47492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:24.464314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1194/2382] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:24.472530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a2caf-9b8e-4c56-aa20-7dce4b3a7316 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:24.472993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a2caf-9b8e-4c56-aa20-7dce4b3a7316 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:24.473274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a2caf-9b8e-4c56-aa20-7dce4b3a7316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:24.500541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-226a2caf-9b8e-4c56-aa20-7dce4b3a7316 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:24.501747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-226a2caf-9b8e-4c56-aa20-7dce4b3a7316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:24.509762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1189/2383] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:24.514168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434237d-23db-45b3-a1b5-d23267c47492 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:24.524660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434237d-23db-45b3-a1b5-d23267c47492 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:24.530024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a3d7b-6403-45c1-8c28-b4c6d155caa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:24.531370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a3d7b-6403-45c1-8c28-b4c6d155caa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:24.531643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a3d7b-6403-45c1-8c28-b4c6d155caa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:24.531980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a3d7b-6403-45c1-8c28-b4c6d155caa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:24.532834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a3d7b-6403-45c1-8c28-b4c6d155caa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:15:24.533133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a3d7b-6403-45c1-8c28-b4c6d155caa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:24.534729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a3d7b-6403-45c1-8c28-b4c6d155caa5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:24.547596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1195/2384] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:15:24.552284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a2caf-9b8e-4c56-aa20-7dce4b3a7316 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:24.562359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a2caf-9b8e-4c56-aa20-7dce4b3a7316 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:24.566989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df217d0b-eea6-4dff-b7b7-b332095f72c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:24.567585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df217d0b-eea6-4dff-b7b7-b332095f72c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:24.567861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df217d0b-eea6-4dff-b7b7-b332095f72c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:24.568128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df217d0b-eea6-4dff-b7b7-b332095f72c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:24.568780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df217d0b-eea6-4dff-b7b7-b332095f72c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:24.570675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df217d0b-eea6-4dff-b7b7-b332095f72c0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:24.653655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1190/2385] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:15:24 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:24.657936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a3d7b-6403-45c1-8c28-b4c6d155caa5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:24.668404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a3d7b-6403-45c1-8c28-b4c6d155caa5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:24.728331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbf575-46db-4f4d-9dce-b628c46bfc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:24.728991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbf575-46db-4f4d-9dce-b628c46bfc7d admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:24.729267 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbf575-46db-4f4d-9dce-b628c46bfc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:24.729536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbf575-46db-4f4d-9dce-b628c46bfc7d admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:24.730158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbf575-46db-4f4d-9dce-b628c46bfc7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:24.732101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbf575-46db-4f4d-9dce-b628c46bfc7d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:24.822015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1196/2386] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:15:24 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:24.823240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df217d0b-eea6-4dff-b7b7-b332095f72c0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:24.834290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df217d0b-eea6-4dff-b7b7-b332095f72c0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:24.838948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e3e4e-d886-4028-a81e-92291b10bdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:24.839527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e3e4e-d886-4028-a81e-92291b10bdc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:24.839796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e3e4e-d886-4028-a81e-92291b10bdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:24.840060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e3e4e-d886-4028-a81e-92291b10bdc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:24.840803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e3e4e-d886-4028-a81e-92291b10bdc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:24.842439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e3e4e-d886-4028-a81e-92291b10bdc4 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:24.856218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1191/2387] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:24.857516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbf575-46db-4f4d-9dce-b628c46bfc7d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:24.869493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbf575-46db-4f4d-9dce-b628c46bfc7d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:24.874647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fb53b-27d8-4472-968c-13039f4a925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:24.875769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fb53b-27d8-4472-968c-13039f4a925c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:24.875769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fb53b-27d8-4472-968c-13039f4a925c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:24.876200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fb53b-27d8-4472-968c-13039f4a925c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:24.876956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fb53b-27d8-4472-968c-13039f4a925c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:24.878647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fb53b-27d8-4472-968c-13039f4a925c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:24.896602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1197/2388] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:15:24.898374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e3e4e-d886-4028-a81e-92291b10bdc4 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:24.911691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e3e4e-d886-4028-a81e-92291b10bdc4 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:24.916918 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b0f47-cc0f-412b-a005-ea5afc99379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:24.917787 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b0f47-cc0f-412b-a005-ea5afc99379c admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:24.918013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b0f47-cc0f-412b-a005-ea5afc99379c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:24.918329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b0f47-cc0f-412b-a005-ea5afc99379c admin admin] PATH_INFO: `/v3/projects/af1141d05df449069acddcf5caf5a512/users/b65df416c9784238bd37c29dde190037/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:24.932098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:24.936444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b0f47-cc0f-412b-a005-ea5afc99379c admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1141d05df449069acddcf5caf5a512, user_id=b65df416c9784238bd37c29dde190037, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:24.939276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b0f47-cc0f-412b-a005-ea5afc99379c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:24.996160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1192/2389] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:15:24 2022] PUT /identity/v3/projects/af1141d05df449069acddcf5caf5a512/users/b65df416c9784238bd37c29dde190037/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:25.007849 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c068114-6797-4428-8024-15d1713ec131 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:25.008134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c068114-6797-4428-8024-15d1713ec131 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:25.008407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c068114-6797-4428-8024-15d1713ec131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:25.039677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9c068114-6797-4428-8024-15d1713ec131 None None] MFA Rules not processed for user `b65df416c9784238bd37c29dde190037`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:25.041209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9c068114-6797-4428-8024-15d1713ec131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:25.077854 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:25.147963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1198/2390] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:25.159552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b0f47-cc0f-412b-a005-ea5afc99379c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:25.161452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-4a5b0f47-cc0f-412b-a005-ea5afc99379c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:25.260943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b0f47-cc0f-412b-a005-ea5afc99379c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:25.265497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459482f-f770-468d-bcca-bc5445255398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:25.266365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459482f-f770-468d-bcca-bc5445255398 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:25.266720 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459482f-f770-468d-bcca-bc5445255398 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:25.266987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459482f-f770-468d-bcca-bc5445255398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:25.281121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459482f-f770-468d-bcca-bc5445255398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:25.282594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459482f-f770-468d-bcca-bc5445255398 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:25.293069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1193/2391] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:27.046273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06caaadb-5e36-49dc-85da-5fac2614abd9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:27.046852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06caaadb-5e36-49dc-85da-5fac2614abd9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:27.046852 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06caaadb-5e36-49dc-85da-5fac2614abd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:27.069900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-06caaadb-5e36-49dc-85da-5fac2614abd9 None None] MFA Rules not processed for user `b65df416c9784238bd37c29dde190037`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:27.070111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-06caaadb-5e36-49dc-85da-5fac2614abd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:27.077950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1199/2392] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:35.297874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:36.464704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f598c-4a2c-4782-9ce5-b07cf5341729 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:36.464704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f598c-4a2c-4782-9ce5-b07cf5341729 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:36.464704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f598c-4a2c-4782-9ce5-b07cf5341729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:36.481483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-053f598c-4a2c-4782-9ce5-b07cf5341729 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:36.482706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-053f598c-4a2c-4782-9ce5-b07cf5341729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:36.551224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1194/2393] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 18:15:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:15:36.566332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06caaadb-5e36-49dc-85da-5fac2614abd9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:36.568394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-06caaadb-5e36-49dc-85da-5fac2614abd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:36.655504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06caaadb-5e36-49dc-85da-5fac2614abd9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:36.662054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275701dd-9035-4ccc-bae3-bae8f9d07401 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:36.662810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275701dd-9035-4ccc-bae3-bae8f9d07401 service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:36.663094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275701dd-9035-4ccc-bae3-bae8f9d07401 service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:36.663397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275701dd-9035-4ccc-bae3-bae8f9d07401 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:36.675122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275701dd-9035-4ccc-bae3-bae8f9d07401 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:36.676769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275701dd-9035-4ccc-bae3-bae8f9d07401 service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:36.695574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1200/2394] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:15:46.543407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=25 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:46.692014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=51 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:49.496183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c0a36-2c06-4270-a9dd-663e20ec40f6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:49.496697 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c0a36-2c06-4270-a9dd-663e20ec40f6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:49.496838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c0a36-2c06-4270-a9dd-663e20ec40f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:49.516023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1f4c0a36-2c06-4270-a9dd-663e20ec40f6 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:49.517234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1f4c0a36-2c06-4270-a9dd-663e20ec40f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:49.524312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1195/2395] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:15:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:49.536910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275701dd-9035-4ccc-bae3-bae8f9d07401 service swift] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:49.547169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275701dd-9035-4ccc-bae3-bae8f9d07401 service swift] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:49.552248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29129de8-000f-4a70-b613-7058e951d602 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:49.552858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29129de8-000f-4a70-b613-7058e951d602 service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:49.553130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29129de8-000f-4a70-b613-7058e951d602 service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:49.553420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29129de8-000f-4a70-b613-7058e951d602 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:49.561805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29129de8-000f-4a70-b613-7058e951d602 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:49.563382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29129de8-000f-4a70-b613-7058e951d602 service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:49.571109 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1201/2396] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:15:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:15:50.590023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e88c9-fd7e-4321-8681-826cf4ea5e7f None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:50.590560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e88c9-fd7e-4321-8681-826cf4ea5e7f None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:50.590560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e88c9-fd7e-4321-8681-826cf4ea5e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:50.612457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-0a0e88c9-fd7e-4321-8681-826cf4ea5e7f None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:50.613719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-0a0e88c9-fd7e-4321-8681-826cf4ea5e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:50.622225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1196/2397] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 18:15:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:50.638600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29129de8-000f-4a70-b613-7058e951d602 service swift] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:50.651223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29129de8-000f-4a70-b613-7058e951d602 service swift] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:50.656916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cbedf-5c51-4690-838f-2c9a533901b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:50.657522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cbedf-5c51-4690-838f-2c9a533901b6 service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:50.657801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cbedf-5c51-4690-838f-2c9a533901b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:50.658076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cbedf-5c51-4690-838f-2c9a533901b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:50.666883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cbedf-5c51-4690-838f-2c9a533901b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:50.668484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cbedf-5c51-4690-838f-2c9a533901b6 service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:50.677467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1202/2398] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:15:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:15:51.354172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582eba-9868-45cf-a5e2-c6d856fa52c3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:51.354507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582eba-9868-45cf-a5e2-c6d856fa52c3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:51.354782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0582eba-9868-45cf-a5e2-c6d856fa52c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:51.374172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c0582eba-9868-45cf-a5e2-c6d856fa52c3 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:51.375379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c0582eba-9868-45cf-a5e2-c6d856fa52c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:51.383027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1197/2399] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:15:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:51.395551 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cbedf-5c51-4690-838f-2c9a533901b6 service swift] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:51.406315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cbedf-5c51-4690-838f-2c9a533901b6 service swift] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:51.411596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32fbac-d274-4f31-a96e-269e1a044e15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:51.412252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32fbac-d274-4f31-a96e-269e1a044e15 service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:51.412553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32fbac-d274-4f31-a96e-269e1a044e15 service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:51.412829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32fbac-d274-4f31-a96e-269e1a044e15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:51.421436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32fbac-d274-4f31-a96e-269e1a044e15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:51.423068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32fbac-d274-4f31-a96e-269e1a044e15 service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:51.430963 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1203/2400] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:15:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:15:51.978244 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03150cc1-4834-49ff-9c4f-a04d8ab5f6de None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:51.978658 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03150cc1-4834-49ff-9c4f-a04d8ab5f6de None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:51.978957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03150cc1-4834-49ff-9c4f-a04d8ab5f6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:52.001263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-03150cc1-4834-49ff-9c4f-a04d8ab5f6de None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:52.002550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-03150cc1-4834-49ff-9c4f-a04d8ab5f6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:52.011191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1198/2401] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 18:15:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:52.023914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32fbac-d274-4f31-a96e-269e1a044e15 service swift] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:52.038150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32fbac-d274-4f31-a96e-269e1a044e15 service swift] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:52.042961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb28889-68ba-4875-9317-6f04980a7cef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:52.043555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb28889-68ba-4875-9317-6f04980a7cef service swift] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:52.043893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb28889-68ba-4875-9317-6f04980a7cef service swift] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:52.044225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb28889-68ba-4875-9317-6f04980a7cef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:52.053009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb28889-68ba-4875-9317-6f04980a7cef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:52.054635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb28889-68ba-4875-9317-6f04980a7cef service swift] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:52.063240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1204/2402] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:15:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:15:52.437534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03150cc1-4834-49ff-9c4f-a04d8ab5f6de None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:52.438894 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-03150cc1-4834-49ff-9c4f-a04d8ab5f6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:52.502068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03150cc1-4834-49ff-9c4f-a04d8ab5f6de None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:52.505618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c1e87-f6a4-4770-98ce-313899ff4f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:52.506215 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c1e87-f6a4-4770-98ce-313899ff4f1a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:52.506482 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c1e87-f6a4-4770-98ce-313899ff4f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:52.506771 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c1e87-f6a4-4770-98ce-313899ff4f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:52.508522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-123c1e87-f6a4-4770-98ce-313899ff4f1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:52.578654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c1e87-f6a4-4770-98ce-313899ff4f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:52.580389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c1e87-f6a4-4770-98ce-313899ff4f1a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:52.590503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1199/2403] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:54.802409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb28889-68ba-4875-9317-6f04980a7cef service swift] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:54.813363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb28889-68ba-4875-9317-6f04980a7cef service swift] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:54.817881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:54.818493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:54.818765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:54.819035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] PATH_INFO: `/v3/users/b65df416c9784238bd37c29dde190037` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:54.820146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65df416c9784238bd37c29dde190037)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:54.822122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:54.984151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c1e87-f6a4-4770-98ce-313899ff4f1a service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:54.996784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c1e87-f6a4-4770-98ce-313899ff4f1a service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:55.002304 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56342f-1c58-4a76-afe2-d996def10e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:55.002905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56342f-1c58-4a76-afe2-d996def10e1c service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:55.003187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56342f-1c58-4a76-afe2-d996def10e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:55.003436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56342f-1c58-4a76-afe2-d996def10e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:55.008612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1e56342f-1c58-4a76-afe2-d996def10e1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:55.010153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65df416c9784238bd37c29dde190037'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.021193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56342f-1c58-4a76-afe2-d996def10e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:55.022788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56342f-1c58-4a76-afe2-d996def10e1c service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:55.033323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1200/2404] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:55.091649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65df416c9784238bd37c29dde190037'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.099482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65df416c9784238bd37c29dde190037'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.100140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65df416c9784238bd37c29dde190037'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.100772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65df416c9784238bd37c29dde190037'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.121086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1205/2405] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:15:54 2022] DELETE /identity/v3/users/b65df416c9784238bd37c29dde190037 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:55.126079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3939b09-dce4-46d6-86b8-9231a4cf5550 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:55.126396 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3939b09-dce4-46d6-86b8-9231a4cf5550 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:55.126671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3939b09-dce4-46d6-86b8-9231a4cf5550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:55.146544 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e3939b09-dce4-46d6-86b8-9231a4cf5550 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:55.147746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e3939b09-dce4-46d6-86b8-9231a4cf5550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:55.210160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1201/2406] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:55.220133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:55.221591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:55.280544 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1df13-59b4-4fbe-b8c3-baaa9a93aade admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:55.284184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f934c-cb8c-4add-99ea-a7dc88389435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:55.284976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f934c-cb8c-4add-99ea-a7dc88389435 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:55.285256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f934c-cb8c-4add-99ea-a7dc88389435 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:55.285528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f934c-cb8c-4add-99ea-a7dc88389435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:55.296204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f934c-cb8c-4add-99ea-a7dc88389435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:55.297690 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f934c-cb8c-4add-99ea-a7dc88389435 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:55.307332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1206/2407] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:55.554166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3939b09-dce4-46d6-86b8-9231a4cf5550 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:55.555690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e3939b09-dce4-46d6-86b8-9231a4cf5550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:55.593570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3939b09-dce4-46d6-86b8-9231a4cf5550 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:55.599643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:55.601243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:55.601995 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:55.602471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] PATH_INFO: `/v3/projects/af1141d05df449069acddcf5caf5a512` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:55.603974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1141d05df449069acddcf5caf5a512)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:55.606256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:55.613344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:55.731167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1141d05df449069acddcf5caf5a512', 'request_id': 'req-4066ea17-0e73-4f09-b29b-429924c23d8a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.738341 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1141d05df449069acddcf5caf5a512', 'request_id': 'req-4066ea17-0e73-4f09-b29b-429924c23d8a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.823997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1141d05df449069acddcf5caf5a512', 'request_id': 'req-4066ea17-0e73-4f09-b29b-429924c23d8a'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.836795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] The token cache is being invalidate because project af1141d05df449069acddcf5caf5a512 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:15:55.837087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.843536 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-4066ea17-0e73-4f09-b29b-429924c23d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:55.845130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1202/2408] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:15:55 2022] DELETE /identity/v3/projects/af1141d05df449069acddcf5caf5a512 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:55.850136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f0ba-72e9-4e90-8ba3-77a03c76c79f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:55.850644 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f0ba-72e9-4e90-8ba3-77a03c76c79f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:55.850745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f0ba-72e9-4e90-8ba3-77a03c76c79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:55.870289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-33b9f0ba-72e9-4e90-8ba3-77a03c76c79f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:55.871511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-33b9f0ba-72e9-4e90-8ba3-77a03c76c79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:55.889339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:55.944614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1207/2409] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:55.995850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5a006-4a18-410c-84b5-a5bd02ca7726 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:55.996349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5a006-4a18-410c-84b5-a5bd02ca7726 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:55.996464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5a006-4a18-410c-84b5-a5bd02ca7726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:56.016425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-36e5a006-4a18-410c-84b5-a5bd02ca7726 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:56.017713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-36e5a006-4a18-410c-84b5-a5bd02ca7726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:56.025940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1203/2410] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:56.030106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f0ba-72e9-4e90-8ba3-77a03c76c79f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:56.040539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f0ba-72e9-4e90-8ba3-77a03c76c79f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:56.056964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17655eda-ffba-4ed0-890c-8d8070b0f58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:56.057779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655eda-ffba-4ed0-890c-8d8070b0f58c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:56.058073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655eda-ffba-4ed0-890c-8d8070b0f58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:56.058336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655eda-ffba-4ed0-890c-8d8070b0f58c admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:56.058983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17655eda-ffba-4ed0-890c-8d8070b0f58c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:15:56.059277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17655eda-ffba-4ed0-890c-8d8070b0f58c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:56.060906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17655eda-ffba-4ed0-890c-8d8070b0f58c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:56.073927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1208/2411] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:15:56.078449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5a006-4a18-410c-84b5-a5bd02ca7726 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:56.097464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5a006-4a18-410c-84b5-a5bd02ca7726 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:56.102592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f866bb-59a2-4598-b41a-6c66f4f0a748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:56.103214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f866bb-59a2-4598-b41a-6c66f4f0a748 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:56.103469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f866bb-59a2-4598-b41a-6c66f4f0a748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:56.103733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f866bb-59a2-4598-b41a-6c66f4f0a748 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:56.104344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f866bb-59a2-4598-b41a-6c66f4f0a748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:56.106790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f866bb-59a2-4598-b41a-6c66f4f0a748 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:56.266494 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1204/2412] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:15:56 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:15:56.271054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17655eda-ffba-4ed0-890c-8d8070b0f58c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:56.284355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17655eda-ffba-4ed0-890c-8d8070b0f58c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:56.351024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083a1fc-5d8e-42a2-9b93-c319272c7c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:56.351904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a1fc-5d8e-42a2-9b93-c319272c7c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:56.352356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a1fc-5d8e-42a2-9b93-c319272c7c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:56.352791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a1fc-5d8e-42a2-9b93-c319272c7c32 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:56.353645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083a1fc-5d8e-42a2-9b93-c319272c7c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:56.355751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083a1fc-5d8e-42a2-9b93-c319272c7c32 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:56.473817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1209/2413] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:15:56 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:56.475175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f866bb-59a2-4598-b41a-6c66f4f0a748 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:56.487481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f866bb-59a2-4598-b41a-6c66f4f0a748 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:56.492672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafd4e2-ff6d-4291-8cc8-3608dfa74abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:56.493387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafd4e2-ff6d-4291-8cc8-3608dfa74abf admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:56.493772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafd4e2-ff6d-4291-8cc8-3608dfa74abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:56.494240 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafd4e2-ff6d-4291-8cc8-3608dfa74abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:56.494994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafd4e2-ff6d-4291-8cc8-3608dfa74abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:56.497225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafd4e2-ff6d-4291-8cc8-3608dfa74abf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:56.511695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1205/2414] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:56.513697 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083a1fc-5d8e-42a2-9b93-c319272c7c32 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:56.527087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083a1fc-5d8e-42a2-9b93-c319272c7c32 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:56.533464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78392b-0e22-4cf8-a12a-387d5090a616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:56.534786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78392b-0e22-4cf8-a12a-387d5090a616 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:56.535000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78392b-0e22-4cf8-a12a-387d5090a616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:56.535271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78392b-0e22-4cf8-a12a-387d5090a616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:56.535939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78392b-0e22-4cf8-a12a-387d5090a616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:56.538094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78392b-0e22-4cf8-a12a-387d5090a616 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:56.550522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1210/2415] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:56.554870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafd4e2-ff6d-4291-8cc8-3608dfa74abf admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:56.568675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafd4e2-ff6d-4291-8cc8-3608dfa74abf admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:56.575651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d806b-ba75-454e-97be-07891f50741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:56.576378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d806b-ba75-454e-97be-07891f50741d admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:56.582340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d806b-ba75-454e-97be-07891f50741d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:56.582340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d806b-ba75-454e-97be-07891f50741d admin admin] PATH_INFO: `/v3/projects/c98de54846de4169b0ea5cca7aae3cf0/users/0c911d938fd64f25844903809d56b263/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:56.594528 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d806b-ba75-454e-97be-07891f50741d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c98de54846de4169b0ea5cca7aae3cf0, user_id=0c911d938fd64f25844903809d56b263, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:56.598660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d806b-ba75-454e-97be-07891f50741d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:56.728051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1206/2416] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:15:56 2022] PUT /identity/v3/projects/c98de54846de4169b0ea5cca7aae3cf0/users/0c911d938fd64f25844903809d56b263/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:56.730165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17670e2-6dd7-4ddb-b733-0856e7532ef2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:56.730645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17670e2-6dd7-4ddb-b733-0856e7532ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:56.731208 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17670e2-6dd7-4ddb-b733-0856e7532ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:56.754475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b17670e2-6dd7-4ddb-b733-0856e7532ef2 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:56.755757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b17670e2-6dd7-4ddb-b733-0856e7532ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:56.837705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1211/2417] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:56.848151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d806b-ba75-454e-97be-07891f50741d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:56.850067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-111d806b-ba75-454e-97be-07891f50741d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:56.890789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17670e2-6dd7-4ddb-b733-0856e7532ef2 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:56.893939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b17670e2-6dd7-4ddb-b733-0856e7532ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:56.933011 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d806b-ba75-454e-97be-07891f50741d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:56.937457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd484d-192f-4bfc-a8ab-5b65b071fa0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:56.938160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd484d-192f-4bfc-a8ab-5b65b071fa0a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:56.938440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd484d-192f-4bfc-a8ab-5b65b071fa0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:56.938714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd484d-192f-4bfc-a8ab-5b65b071fa0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:56.942189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:56.949434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd484d-192f-4bfc-a8ab-5b65b071fa0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:56.950889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd484d-192f-4bfc-a8ab-5b65b071fa0a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:56.962711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1207/2418] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:15:56.996965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17670e2-6dd7-4ddb-b733-0856e7532ef2 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:57.002313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0172d8-fa95-438c-b89c-f11580114d83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:57.002914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0172d8-fa95-438c-b89c-f11580114d83 service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:57.003189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0172d8-fa95-438c-b89c-f11580114d83 service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:57.003455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0172d8-fa95-438c-b89c-f11580114d83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:57.005237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ac0172d8-fa95-438c-b89c-f11580114d83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:57.017390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0172d8-fa95-438c-b89c-f11580114d83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:57.018973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0172d8-fa95-438c-b89c-f11580114d83 service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:57.028890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1212/2419] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:15:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:15:57.087884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd484d-192f-4bfc-a8ab-5b65b071fa0a service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:57.089351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1cdd484d-192f-4bfc-a8ab-5b65b071fa0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:57.103280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd484d-192f-4bfc-a8ab-5b65b071fa0a service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:57.108282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:57.108992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:57.109258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:57.109711 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] PATH_INFO: `/v3/users/caf8d88ca4da4515bd9f1bec64da3375` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:57.111115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf8d88ca4da4515bd9f1bec64da3375)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:57.113229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:57.432369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:57.555044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf8d88ca4da4515bd9f1bec64da3375'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:57.657495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf8d88ca4da4515bd9f1bec64da3375'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:57.665898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf8d88ca4da4515bd9f1bec64da3375'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:57.666239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf8d88ca4da4515bd9f1bec64da3375'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:57.666937 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-c53303ab-bc49-481e-87cd-d2233070c781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf8d88ca4da4515bd9f1bec64da3375'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:57.687350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1208/2420] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:15:57 2022] DELETE /identity/v3/users/caf8d88ca4da4515bd9f1bec64da3375 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:58.429049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0172d8-fa95-438c-b89c-f11580114d83 service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:58.440239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0172d8-fa95-438c-b89c-f11580114d83 service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:58.505063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:58.505607 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:58.505889 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:58.506159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] PATH_INFO: `/v3/projects/5de58eb192de494b809a04e638e2a291` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:58.507275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de58eb192de494b809a04e638e2a291)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:58.509888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:58.627748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de58eb192de494b809a04e638e2a291', 'request_id': 'req-852ee75a-c0f1-4cdc-8468-924dd5088efb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:58.634227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de58eb192de494b809a04e638e2a291', 'request_id': 'req-852ee75a-c0f1-4cdc-8468-924dd5088efb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:58.732771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de58eb192de494b809a04e638e2a291', 'request_id': 'req-852ee75a-c0f1-4cdc-8468-924dd5088efb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:58.747072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] The token cache is being invalidate because project 5de58eb192de494b809a04e638e2a291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:15:58.747356 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:58.748441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-852ee75a-c0f1-4cdc-8468-924dd5088efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:15:58.751072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1213/2421] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:15:58 2022] DELETE /identity/v3/projects/5de58eb192de494b809a04e638e2a291 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:58.756111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eda1a7-a406-4dca-aa66-aa2e4ae9c722 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:58.756634 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eda1a7-a406-4dca-aa66-aa2e4ae9c722 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:58.757002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eda1a7-a406-4dca-aa66-aa2e4ae9c722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:58.802618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d0eda1a7-a406-4dca-aa66-aa2e4ae9c722 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:58.804872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d0eda1a7-a406-4dca-aa66-aa2e4ae9c722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:58.893786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1209/2422] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:58.901971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e22f21-8b2f-4ad4-8dd4-20a7a923d9a0 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:58.902575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e22f21-8b2f-4ad4-8dd4-20a7a923d9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:58.902804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e22f21-8b2f-4ad4-8dd4-20a7a923d9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:58.926150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-62e22f21-8b2f-4ad4-8dd4-20a7a923d9a0 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:58.929525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-62e22f21-8b2f-4ad4-8dd4-20a7a923d9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:58.958954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1214/2423] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:58.960816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eda1a7-a406-4dca-aa66-aa2e4ae9c722 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:58.974849 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eda1a7-a406-4dca-aa66-aa2e4ae9c722 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:58.992814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd8d9f-46fe-46fb-90cc-3a16bfbac3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:58.993779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd8d9f-46fe-46fb-90cc-3a16bfbac3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:58.994763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd8d9f-46fe-46fb-90cc-3a16bfbac3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:58.995217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd8d9f-46fe-46fb-90cc-3a16bfbac3fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:58.996059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd8d9f-46fe-46fb-90cc-3a16bfbac3fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:15:58.996555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd8d9f-46fe-46fb-90cc-3a16bfbac3fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:59.001633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd8d9f-46fe-46fb-90cc-3a16bfbac3fd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:59.021238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1210/2424] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:15:59.023972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e22f21-8b2f-4ad4-8dd4-20a7a923d9a0 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:59.033744 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e22f21-8b2f-4ad4-8dd4-20a7a923d9a0 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:59.040558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4b958-d44c-4cf6-8e1b-85b81d02f18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:59.041174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4b958-d44c-4cf6-8e1b-85b81d02f18c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:59.041450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4b958-d44c-4cf6-8e1b-85b81d02f18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:59.041730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4b958-d44c-4cf6-8e1b-85b81d02f18c admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:59.042351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4b958-d44c-4cf6-8e1b-85b81d02f18c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:59.044370 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4b958-d44c-4cf6-8e1b-85b81d02f18c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:59.167145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1215/2425] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:15:59 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:59.168267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd8d9f-46fe-46fb-90cc-3a16bfbac3fd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:59.181856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd8d9f-46fe-46fb-90cc-3a16bfbac3fd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:59.251989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6cef7-f937-4662-994d-f0160ca35c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:59.252631 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6cef7-f937-4662-994d-f0160ca35c00 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:59.253371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6cef7-f937-4662-994d-f0160ca35c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:59.253649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6cef7-f937-4662-994d-f0160ca35c00 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:59.254425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6cef7-f937-4662-994d-f0160ca35c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:59.256920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6cef7-f937-4662-994d-f0160ca35c00 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:59.360400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1211/2426] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:15:59 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:15:59.364559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4b958-d44c-4cf6-8e1b-85b81d02f18c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:59.384390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4b958-d44c-4cf6-8e1b-85b81d02f18c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:59.393498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f890a-7cc8-41a7-80eb-bb40b8c86143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:59.395469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f890a-7cc8-41a7-80eb-bb40b8c86143 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:59.395978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f890a-7cc8-41a7-80eb-bb40b8c86143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:59.396414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f890a-7cc8-41a7-80eb-bb40b8c86143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:59.397108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f890a-7cc8-41a7-80eb-bb40b8c86143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:59.398741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f890a-7cc8-41a7-80eb-bb40b8c86143 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:59.424073 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1216/2427] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:59.424214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6cef7-f937-4662-994d-f0160ca35c00 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:59.430496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6cef7-f937-4662-994d-f0160ca35c00 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:59.438795 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dce648-9578-4f8b-9336-61c8c70dcb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:59.439540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dce648-9578-4f8b-9336-61c8c70dcb48 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:59.439844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dce648-9578-4f8b-9336-61c8c70dcb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:59.440104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dce648-9578-4f8b-9336-61c8c70dcb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:59.440792 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dce648-9578-4f8b-9336-61c8c70dcb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:59.442452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dce648-9578-4f8b-9336-61c8c70dcb48 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:59.455168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1212/2428] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:15:59.459511 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f890a-7cc8-41a7-80eb-bb40b8c86143 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:59.471306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f890a-7cc8-41a7-80eb-bb40b8c86143 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:59.476490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e07fac-51f9-4368-9f45-6c05009d157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:59.477196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e07fac-51f9-4368-9f45-6c05009d157d admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:59.477474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e07fac-51f9-4368-9f45-6c05009d157d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:59.477747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e07fac-51f9-4368-9f45-6c05009d157d admin admin] PATH_INFO: `/v3/projects/b6546b87f4d74110825d3554551690d7/users/d9a66e015d5d418bb0e95e3570059b80/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:59.481887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:59.497815 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e07fac-51f9-4368-9f45-6c05009d157d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6546b87f4d74110825d3554551690d7, user_id=d9a66e015d5d418bb0e95e3570059b80, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:59.501139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e07fac-51f9-4368-9f45-6c05009d157d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:15:59.584969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1217/2429] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:15:59 2022] PUT /identity/v3/projects/b6546b87f4d74110825d3554551690d7/users/d9a66e015d5d418bb0e95e3570059b80/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:15:59.598013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ad278-d5db-4c37-b6ef-fa512b696b72 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:59.598796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ad278-d5db-4c37-b6ef-fa512b696b72 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:59.599161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ad278-d5db-4c37-b6ef-fa512b696b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:59.622057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:15:59.631125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-cf4ad278-d5db-4c37-b6ef-fa512b696b72 None None] MFA Rules not processed for user `d9a66e015d5d418bb0e95e3570059b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:15:59.632840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-cf4ad278-d5db-4c37-b6ef-fa512b696b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:15:59.726354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1213/2430] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:15:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:15:59.730696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e07fac-51f9-4368-9f45-6c05009d157d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:15:59.904642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e07fac-51f9-4368-9f45-6c05009d157d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:15:59.914764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59c00-1d79-4993-8624-81f3116d9b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:15:59.915614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59c00-1d79-4993-8624-81f3116d9b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:15:59.916542 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59c00-1d79-4993-8624-81f3116d9b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:15:59.919380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59c00-1d79-4993-8624-81f3116d9b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:15:59.920399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce59c00-1d79-4993-8624-81f3116d9b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:15:59.923629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce59c00-1d79-4993-8624-81f3116d9b4d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:00.032452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1218/2431] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:15:59 2022] POST /identity/v3/projects => generated 373 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:00.036031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ad278-d5db-4c37-b6ef-fa512b696b72 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:00.053977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ad278-d5db-4c37-b6ef-fa512b696b72 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:00.127104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5e8fa-97e5-47ec-a9ec-879e4891af89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:00.127710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5e8fa-97e5-47ec-a9ec-879e4891af89 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:00.128623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5e8fa-97e5-47ec-a9ec-879e4891af89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:00.128623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5e8fa-97e5-47ec-a9ec-879e4891af89 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:00.129029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5e8fa-97e5-47ec-a9ec-879e4891af89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:00.130963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5e8fa-97e5-47ec-a9ec-879e4891af89 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:00.156896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59c00-1d79-4993-8624-81f3116d9b4d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:00.163255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fce59c00-1d79-4993-8624-81f3116d9b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:00.171967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59c00-1d79-4993-8624-81f3116d9b4d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:00.180588 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:00.181230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:00.181571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:00.182154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] PATH_INFO: `/v3/users/2d5e7d5c77084705b1f39406293ab585` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:00.183122 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5e7d5c77084705b1f39406293ab585)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:00.185220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:00.273625 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1214/2432] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:00 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:00.275034 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5e8fa-97e5-47ec-a9ec-879e4891af89 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:00.284929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5e8fa-97e5-47ec-a9ec-879e4891af89 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:00.290397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b96db8-e3ae-49cc-a5a1-1c71ffac09c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:00.293449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b96db8-e3ae-49cc-a5a1-1c71ffac09c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:00.293449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b96db8-e3ae-49cc-a5a1-1c71ffac09c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:00.293699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b96db8-e3ae-49cc-a5a1-1c71ffac09c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:00.294220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b96db8-e3ae-49cc-a5a1-1c71ffac09c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:00.295899 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b96db8-e3ae-49cc-a5a1-1c71ffac09c1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:00.306065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1215/2433] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:00.310284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b96db8-e3ae-49cc-a5a1-1c71ffac09c1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:00.324349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b96db8-e3ae-49cc-a5a1-1c71ffac09c1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:00.330444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b30add-3ef8-419f-87e2-e8ea081e53ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:00.336690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b30add-3ef8-419f-87e2-e8ea081e53ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:00.336690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b30add-3ef8-419f-87e2-e8ea081e53ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:00.336690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b30add-3ef8-419f-87e2-e8ea081e53ae admin admin] PATH_INFO: `/v3/projects/067cd894c05f4f7ea32fb3ca45dabac7/users/1a05697db44a45799043872941847353/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:00.351481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b30add-3ef8-419f-87e2-e8ea081e53ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=067cd894c05f4f7ea32fb3ca45dabac7, user_id=1a05697db44a45799043872941847353, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:00.354628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b30add-3ef8-419f-87e2-e8ea081e53ae admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:00.410035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5e7d5c77084705b1f39406293ab585'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:00.476036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1216/2434] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:00 2022] PUT /identity/v3/projects/067cd894c05f4f7ea32fb3ca45dabac7/users/1a05697db44a45799043872941847353/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:00.481134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b30add-3ef8-419f-87e2-e8ea081e53ae admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:00.500727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5e7d5c77084705b1f39406293ab585'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:00.513774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5e7d5c77084705b1f39406293ab585'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:00.516227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5e7d5c77084705b1f39406293ab585'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:00.517082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5e7d5c77084705b1f39406293ab585'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:00.555290 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1219/2435] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:00 2022] DELETE /identity/v3/users/2d5e7d5c77084705b1f39406293ab585 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:00.588682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b30add-3ef8-419f-87e2-e8ea081e53ae admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:00.677581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200818d7-6a74-4658-8141-f65f42b63e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:00.678316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200818d7-6a74-4658-8141-f65f42b63e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:00.678869 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200818d7-6a74-4658-8141-f65f42b63e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:00.679225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200818d7-6a74-4658-8141-f65f42b63e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:00.680033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200818d7-6a74-4658-8141-f65f42b63e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:00.685123 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200818d7-6a74-4658-8141-f65f42b63e4f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:00.705187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1217/2436] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:00.710056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:00.714525 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:00.728337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22c3ad-88a6-40f1-819e-c6445a13dbff admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:00.734831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f6849-640d-4b6b-bc5d-789484f968c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:00.735734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f6849-640d-4b6b-bc5d-789484f968c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:00.736016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f6849-640d-4b6b-bc5d-789484f968c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:00.736293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f6849-640d-4b6b-bc5d-789484f968c4 admin admin] PATH_INFO: `/v3/projects/067cd894c05f4f7ea32fb3ca45dabac7/users/1a05697db44a45799043872941847353/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:00.739536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f6849-640d-4b6b-bc5d-789484f968c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=067cd894c05f4f7ea32fb3ca45dabac7, user_id=1a05697db44a45799043872941847353, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:00.742445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f6849-640d-4b6b-bc5d-789484f968c4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:00.754038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1220/2437] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:00 2022] PUT /identity/v3/projects/067cd894c05f4f7ea32fb3ca45dabac7/users/1a05697db44a45799043872941847353/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:00.765703 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce664f-14ac-4904-9ec2-5c2a3f6a13e2 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:00.766060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce664f-14ac-4904-9ec2-5c2a3f6a13e2 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:00.766492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce664f-14ac-4904-9ec2-5c2a3f6a13e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:00.813199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6cce664f-14ac-4904-9ec2-5c2a3f6a13e2 None None] MFA Rules not processed for user `1a05697db44a45799043872941847353`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:00.814541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6cce664f-14ac-4904-9ec2-5c2a3f6a13e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:00.887711 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f6849-640d-4b6b-bc5d-789484f968c4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:00.891049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ec4f6849-640d-4b6b-bc5d-789484f968c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:00.907066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:00.932645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1218/2438] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:00.941454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce664f-14ac-4904-9ec2-5c2a3f6a13e2 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:00.944388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6cce664f-14ac-4904-9ec2-5c2a3f6a13e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:00.964983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:01.014172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f6849-640d-4b6b-bc5d-789484f968c4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:01.021507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:01.022433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:01.026759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:01.026759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] PATH_INFO: `/v3/projects/c0061f213bfd4e3b87a0582b6cec6487` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:01.026759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0061f213bfd4e3b87a0582b6cec6487)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:01.026759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:01.037626 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce664f-14ac-4904-9ec2-5c2a3f6a13e2 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:01.041712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a90749-d5e9-47c6-818e-e5a268a72935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:01.042454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a90749-d5e9-47c6-818e-e5a268a72935 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:01.042842 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a90749-d5e9-47c6-818e-e5a268a72935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:01.043221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a90749-d5e9-47c6-818e-e5a268a72935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:01.069612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a90749-d5e9-47c6-818e-e5a268a72935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:01.071134 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a90749-d5e9-47c6-818e-e5a268a72935 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:01.092974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1219/2439] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:01.155391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0061f213bfd4e3b87a0582b6cec6487', 'request_id': 'req-90f47772-3a53-43be-9141-ad3556d3dee7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:01.164175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0061f213bfd4e3b87a0582b6cec6487', 'request_id': 'req-90f47772-3a53-43be-9141-ad3556d3dee7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:01.288586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0061f213bfd4e3b87a0582b6cec6487', 'request_id': 'req-90f47772-3a53-43be-9141-ad3556d3dee7'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:01.310487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] The token cache is being invalidate because project c0061f213bfd4e3b87a0582b6cec6487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:01.310779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:01.312030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-90f47772-3a53-43be-9141-ad3556d3dee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:01.313996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1221/2440] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:00 2022] DELETE /identity/v3/projects/c0061f213bfd4e3b87a0582b6cec6487 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:01.319387 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06354458-8633-444e-89fe-b03512e48e3d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:01.319673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06354458-8633-444e-89fe-b03512e48e3d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:01.319987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06354458-8633-444e-89fe-b03512e48e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:01.341794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-06354458-8633-444e-89fe-b03512e48e3d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:01.343037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-06354458-8633-444e-89fe-b03512e48e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:01.438203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1220/2441] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:01.445768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565805e-f9b9-4098-b1d2-12d895e6d726 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:01.448837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565805e-f9b9-4098-b1d2-12d895e6d726 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:01.448837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565805e-f9b9-4098-b1d2-12d895e6d726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:01.474203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3565805e-f9b9-4098-b1d2-12d895e6d726 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:01.474203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3565805e-f9b9-4098-b1d2-12d895e6d726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:01.490898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1222/2442] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:01.491421 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06354458-8633-444e-89fe-b03512e48e3d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:01.502944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06354458-8633-444e-89fe-b03512e48e3d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:01.523621 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59897c6-2e97-4639-8e92-c19a2412a5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:01.524262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59897c6-2e97-4639-8e92-c19a2412a5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:01.524540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59897c6-2e97-4639-8e92-c19a2412a5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:01.524793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59897c6-2e97-4639-8e92-c19a2412a5dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:01.528149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59897c6-2e97-4639-8e92-c19a2412a5dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:16:01.528149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59897c6-2e97-4639-8e92-c19a2412a5dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:01.529897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59897c6-2e97-4639-8e92-c19a2412a5dd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:01.555665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1221/2443] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:16:01.560054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565805e-f9b9-4098-b1d2-12d895e6d726 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:01.577201 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565805e-f9b9-4098-b1d2-12d895e6d726 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:01.582900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a3a6d-62ab-4f16-80a6-de3b11b76cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:01.583645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a3a6d-62ab-4f16-80a6-de3b11b76cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:01.583921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a3a6d-62ab-4f16-80a6-de3b11b76cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:01.584185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a3a6d-62ab-4f16-80a6-de3b11b76cb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:01.584907 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a3a6d-62ab-4f16-80a6-de3b11b76cb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:01.587084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a3a6d-62ab-4f16-80a6-de3b11b76cb4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:01.704365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1223/2444] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:01 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:01.706024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59897c6-2e97-4639-8e92-c19a2412a5dd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:01.721614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59897c6-2e97-4639-8e92-c19a2412a5dd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:01.813057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a55154-1944-4ff9-8b85-a8b5c53fd267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:01.813929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a55154-1944-4ff9-8b85-a8b5c53fd267 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:01.814277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a55154-1944-4ff9-8b85-a8b5c53fd267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:01.814640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a55154-1944-4ff9-8b85-a8b5c53fd267 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:01.815393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a55154-1944-4ff9-8b85-a8b5c53fd267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:01.817897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a55154-1944-4ff9-8b85-a8b5c53fd267 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:01.935930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1222/2445] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:01 2022] POST /identity/v3/users => generated 284 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:01.937280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a3a6d-62ab-4f16-80a6-de3b11b76cb4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:01.947700 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a3a6d-62ab-4f16-80a6-de3b11b76cb4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:01.952739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56f762-f84f-4cd8-94c3-f9abc6fa6a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:01.953328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56f762-f84f-4cd8-94c3-f9abc6fa6a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:01.953748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56f762-f84f-4cd8-94c3-f9abc6fa6a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:01.955532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56f762-f84f-4cd8-94c3-f9abc6fa6a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:01.956306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56f762-f84f-4cd8-94c3-f9abc6fa6a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:01.957935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56f762-f84f-4cd8-94c3-f9abc6fa6a9e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:01.973004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1224/2446] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:01.978658 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a55154-1944-4ff9-8b85-a8b5c53fd267 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:01.991059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a55154-1944-4ff9-8b85-a8b5c53fd267 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:02.011119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7beb8-fe44-403a-b37b-f9e203775472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:02.011119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7beb8-fe44-403a-b37b-f9e203775472 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:02.011119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7beb8-fe44-403a-b37b-f9e203775472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:02.011119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7beb8-fe44-403a-b37b-f9e203775472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:02.011560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7beb8-fe44-403a-b37b-f9e203775472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:02.012777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7beb8-fe44-403a-b37b-f9e203775472 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:02.040048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:02.056564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1223/2447] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:02.057842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56f762-f84f-4cd8-94c3-f9abc6fa6a9e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:02.069627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56f762-f84f-4cd8-94c3-f9abc6fa6a9e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:02.075119 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1365a8-62d0-4bc8-9f91-40a1e33946da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:02.075793 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1365a8-62d0-4bc8-9f91-40a1e33946da admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:02.076067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1365a8-62d0-4bc8-9f91-40a1e33946da admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:02.076332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1365a8-62d0-4bc8-9f91-40a1e33946da admin admin] PATH_INFO: `/v3/projects/d639f04384864de88d1cae7fdf2cb211/users/0f6e81dfe9bb4303b4bce9e6303104f9/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:02.097304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1365a8-62d0-4bc8-9f91-40a1e33946da admin admin] RBAC: Authorizing `identity:create_grant(project_id=d639f04384864de88d1cae7fdf2cb211, user_id=0f6e81dfe9bb4303b4bce9e6303104f9, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:02.103917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1365a8-62d0-4bc8-9f91-40a1e33946da admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:02.195278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1225/2448] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:02 2022] PUT /identity/v3/projects/d639f04384864de88d1cae7fdf2cb211/users/0f6e81dfe9bb4303b4bce9e6303104f9/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:02.205348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c13bee-8d0c-4871-b1a9-1503edfa5b99 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:02.205832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c13bee-8d0c-4871-b1a9-1503edfa5b99 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:02.206270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c13bee-8d0c-4871-b1a9-1503edfa5b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:02.236347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-54c13bee-8d0c-4871-b1a9-1503edfa5b99 None None] MFA Rules not processed for user `0f6e81dfe9bb4303b4bce9e6303104f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:02.238598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-54c13bee-8d0c-4871-b1a9-1503edfa5b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:02.293460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1365a8-62d0-4bc8-9f91-40a1e33946da admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:02.296402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2a1365a8-62d0-4bc8-9f91-40a1e33946da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:02.318922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1224/2449] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:02.327778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c13bee-8d0c-4871-b1a9-1503edfa5b99 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:02.329886 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-54c13bee-8d0c-4871-b1a9-1503edfa5b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:02.418769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1365a8-62d0-4bc8-9f91-40a1e33946da admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:02.421479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c13bee-8d0c-4871-b1a9-1503edfa5b99 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:02.429139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725de4fa-99cc-47f0-bef5-4d2863725672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:02.431689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725de4fa-99cc-47f0-bef5-4d2863725672 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:02.432162 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:02.432451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725de4fa-99cc-47f0-bef5-4d2863725672 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:02.432830 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:02.433107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:02.433800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] PATH_INFO: `/v3/users/d9a66e015d5d418bb0e95e3570059b80` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:02.434094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725de4fa-99cc-47f0-bef5-4d2863725672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:02.434617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9a66e015d5d418bb0e95e3570059b80)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:02.437103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:02.449560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725de4fa-99cc-47f0-bef5-4d2863725672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:02.451355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725de4fa-99cc-47f0-bef5-4d2863725672 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:02.465293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1225/2450] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:02.645252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9a66e015d5d418bb0e95e3570059b80'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:02.681730 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:02.698510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9a66e015d5d418bb0e95e3570059b80'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:02.702146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:02.710276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9a66e015d5d418bb0e95e3570059b80'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:02.711029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9a66e015d5d418bb0e95e3570059b80'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:02.713216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9a66e015d5d418bb0e95e3570059b80'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:02.738226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1226/2451] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:02 2022] DELETE /identity/v3/users/d9a66e015d5d418bb0e95e3570059b80 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:02.742809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be15bf7-bdd2-45c9-8490-d33f6735be88 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:02.743090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be15bf7-bdd2-45c9-8490-d33f6735be88 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:02.743735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be15bf7-bdd2-45c9-8490-d33f6735be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:02.764026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2be15bf7-bdd2-45c9-8490-d33f6735be88 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:02.765837 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2be15bf7-bdd2-45c9-8490-d33f6735be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:02.841575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1226/2452] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:02.855437 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:02.857126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:02.960779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdab9b7-0953-4519-a854-ba33e3bbc741 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:02.965494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cc56c-a270-4039-9034-248d890b6dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:02.966343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cc56c-a270-4039-9034-248d890b6dad service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:02.966736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cc56c-a270-4039-9034-248d890b6dad service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:02.967016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cc56c-a270-4039-9034-248d890b6dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:02.979294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cc56c-a270-4039-9034-248d890b6dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:02.985921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cc56c-a270-4039-9034-248d890b6dad service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:02.997518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1227/2453] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:03.094753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ac288-c427-49d6-b629-00fea7ae619a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:03.095204 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ac288-c427-49d6-b629-00fea7ae619a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:03.095553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ac288-c427-49d6-b629-00fea7ae619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:03.130535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-413ac288-c427-49d6-b629-00fea7ae619a None None] MFA Rules not processed for user `0c911d938fd64f25844903809d56b263`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:03.131809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-413ac288-c427-49d6-b629-00fea7ae619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:03.240386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1227/2454] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:03.241378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cc56c-a270-4039-9034-248d890b6dad service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:03.245569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-085cc56c-a270-4039-9034-248d890b6dad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:03.274421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cc56c-a270-4039-9034-248d890b6dad service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:03.288837 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0b27-15a5-4558-9418-f701de336415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:03.290420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0b27-15a5-4558-9418-f701de336415 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:03.290770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0b27-15a5-4558-9418-f701de336415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:03.291259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0b27-15a5-4558-9418-f701de336415 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:03.292424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a0b27-15a5-4558-9418-f701de336415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:03.294015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a0b27-15a5-4558-9418-f701de336415 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:03.416436 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1228/2455] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:03 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:03.421217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ac288-c427-49d6-b629-00fea7ae619a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:03.433140 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ac288-c427-49d6-b629-00fea7ae619a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:03.494710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc6481-5a48-466d-8d5e-8ce5753d4196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:03.495420 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc6481-5a48-466d-8d5e-8ce5753d4196 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:03.495794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc6481-5a48-466d-8d5e-8ce5753d4196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:03.496131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc6481-5a48-466d-8d5e-8ce5753d4196 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:03.496889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc6481-5a48-466d-8d5e-8ce5753d4196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:03.499172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc6481-5a48-466d-8d5e-8ce5753d4196 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:03.535908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0b27-15a5-4558-9418-f701de336415 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:03.537682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a74a0b27-15a5-4558-9418-f701de336415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:03.553278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0b27-15a5-4558-9418-f701de336415 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:03.558107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:03.558749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:03.559026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:03.559353 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] PATH_INFO: `/v3/projects/b6546b87f4d74110825d3554551690d7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:03.560667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6546b87f4d74110825d3554551690d7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:03.563966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:03.602101 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1228/2456] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:03 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:03.605983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc6481-5a48-466d-8d5e-8ce5753d4196 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:03.617108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc6481-5a48-466d-8d5e-8ce5753d4196 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:03.622108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e942d-237f-4bb4-84b6-3bc75c689aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:03.622725 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e942d-237f-4bb4-84b6-3bc75c689aac admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:03.622998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e942d-237f-4bb4-84b6-3bc75c689aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:03.623263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e942d-237f-4bb4-84b6-3bc75c689aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:03.624463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e942d-237f-4bb4-84b6-3bc75c689aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:03.626088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e942d-237f-4bb4-84b6-3bc75c689aac admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:03.637301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1229/2457] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:03.641526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e942d-237f-4bb4-84b6-3bc75c689aac admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:03.651639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e942d-237f-4bb4-84b6-3bc75c689aac admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:03.656384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7051f59-1f27-4ccc-b4fb-0af8f5e1f351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:03.657080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7051f59-1f27-4ccc-b4fb-0af8f5e1f351 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:03.657355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7051f59-1f27-4ccc-b4fb-0af8f5e1f351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:03.657624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7051f59-1f27-4ccc-b4fb-0af8f5e1f351 admin admin] PATH_INFO: `/v3/projects/01c1922dfd9e47afb271fbfdbc4f0663/users/07fa6d8acbb44b5abac88a7f63fee601/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:03.676639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7051f59-1f27-4ccc-b4fb-0af8f5e1f351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c1922dfd9e47afb271fbfdbc4f0663, user_id=07fa6d8acbb44b5abac88a7f63fee601, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:03.679496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7051f59-1f27-4ccc-b4fb-0af8f5e1f351 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:03.683104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6546b87f4d74110825d3554551690d7', 'request_id': 'req-8264400a-076e-4822-b7ce-2b593a33fa80'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:03.690756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6546b87f4d74110825d3554551690d7', 'request_id': 'req-8264400a-076e-4822-b7ce-2b593a33fa80'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:03.763105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6546b87f4d74110825d3554551690d7', 'request_id': 'req-8264400a-076e-4822-b7ce-2b593a33fa80'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:03.771121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1230/2458] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:03 2022] PUT /identity/v3/projects/01c1922dfd9e47afb271fbfdbc4f0663/users/07fa6d8acbb44b5abac88a7f63fee601/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:03.776223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7051f59-1f27-4ccc-b4fb-0af8f5e1f351 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:03.781281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] The token cache is being invalidate because project b6546b87f4d74110825d3554551690d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:03.781572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:03.782230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:03.783728 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1229/2459] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:03 2022] DELETE /identity/v3/projects/b6546b87f4d74110825d3554551690d7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:03.788219 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:03.794826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:03.824080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:03.864260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7051f59-1f27-4ccc-b4fb-0af8f5e1f351 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:03.874280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f268d-000c-4be2-9110-42af279c8a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:03.875181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f268d-000c-4be2-9110-42af279c8a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:03.875700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f268d-000c-4be2-9110-42af279c8a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:03.876117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f268d-000c-4be2-9110-42af279c8a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:03.879982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f268d-000c-4be2-9110-42af279c8a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:03.882189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f268d-000c-4be2-9110-42af279c8a1e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:03.885536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264400a-076e-4822-b7ce-2b593a33fa80 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:03.895115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:03.899106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:03.899106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:03.899365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1231/2460] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:03.899469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] PATH_INFO: `/v3/users/1a05697db44a45799043872941847353` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:03.899469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a05697db44a45799043872941847353)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:03.900215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:03.900739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f268d-000c-4be2-9110-42af279c8a1e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:03.903138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-231f268d-000c-4be2-9110-42af279c8a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:03.932046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f268d-000c-4be2-9110-42af279c8a1e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:03.937070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568b52c-dfb1-468a-9deb-ad69cb41e3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:03.937832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568b52c-dfb1-468a-9deb-ad69cb41e3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:03.938112 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568b52c-dfb1-468a-9deb-ad69cb41e3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:03.938385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568b52c-dfb1-468a-9deb-ad69cb41e3b6 admin admin] PATH_INFO: `/v3/projects/01c1922dfd9e47afb271fbfdbc4f0663/users/07fa6d8acbb44b5abac88a7f63fee601/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:03.940440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568b52c-dfb1-468a-9deb-ad69cb41e3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c1922dfd9e47afb271fbfdbc4f0663, user_id=07fa6d8acbb44b5abac88a7f63fee601, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:03.943108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568b52c-dfb1-468a-9deb-ad69cb41e3b6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:03.954484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1232/2461] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:03 2022] PUT /identity/v3/projects/01c1922dfd9e47afb271fbfdbc4f0663/users/07fa6d8acbb44b5abac88a7f63fee601/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:04.118488 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a05697db44a45799043872941847353'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:04.190264 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a05697db44a45799043872941847353'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:04.197834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a05697db44a45799043872941847353'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:04.198472 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a05697db44a45799043872941847353'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:04.199085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a05697db44a45799043872941847353'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:04.218477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1230/2462] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:03 2022] DELETE /identity/v3/users/1a05697db44a45799043872941847353 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:04.248843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568b52c-dfb1-468a-9deb-ad69cb41e3b6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:04.252336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a568b52c-dfb1-468a-9deb-ad69cb41e3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:04.328940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568b52c-dfb1-468a-9deb-ad69cb41e3b6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:04.411356 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6995e-8760-43c6-a385-9206cbe5b63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:04.412189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6995e-8760-43c6-a385-9206cbe5b63e service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:04.412896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6995e-8760-43c6-a385-9206cbe5b63e service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:04.413480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6995e-8760-43c6-a385-9206cbe5b63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:04.416155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c0b6995e-8760-43c6-a385-9206cbe5b63e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:04.516665 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:04.529472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6995e-8760-43c6-a385-9206cbe5b63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:04.531563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6995e-8760-43c6-a385-9206cbe5b63e service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:04.546329 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1233/2463] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:04.590133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:04.989902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:04.989902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:05.087649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3090c01-28e9-4327-8180-4ade0fa8fe2e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:05.093613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:05.094550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:05.095083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:05.095768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] PATH_INFO: `/v3/projects/067cd894c05f4f7ea32fb3ca45dabac7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:05.097351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=067cd894c05f4f7ea32fb3ca45dabac7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:05.099670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:05.221586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '067cd894c05f4f7ea32fb3ca45dabac7', 'request_id': 'req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:05.228608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '067cd894c05f4f7ea32fb3ca45dabac7', 'request_id': 'req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:05.333337 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '067cd894c05f4f7ea32fb3ca45dabac7', 'request_id': 'req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:05.347256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] The token cache is being invalidate because project 067cd894c05f4f7ea32fb3ca45dabac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:05.347799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:05.348788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-57bbbd39-ec1c-4f2e-8ec9-64a59a75f1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:05.351586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1231/2464] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:04 2022] DELETE /identity/v3/projects/067cd894c05f4f7ea32fb3ca45dabac7 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:05.358371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b17ee5-b79d-4b9b-880a-ac231914c554 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:05.358858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b17ee5-b79d-4b9b-880a-ac231914c554 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:05.358944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b17ee5-b79d-4b9b-880a-ac231914c554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:05.387158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-38b17ee5-b79d-4b9b-880a-ac231914c554 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:05.388472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-38b17ee5-b79d-4b9b-880a-ac231914c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:05.469734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1234/2465] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:05.477398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba052d-02f9-4a6d-861b-8a1755b0613c None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:05.478042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba052d-02f9-4a6d-861b-8a1755b0613c None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:05.478523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba052d-02f9-4a6d-861b-8a1755b0613c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:05.511358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c5ba052d-02f9-4a6d-861b-8a1755b0613c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:05.512619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c5ba052d-02f9-4a6d-861b-8a1755b0613c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:05.521666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1232/2466] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:05.525706 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b17ee5-b79d-4b9b-880a-ac231914c554 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:05.540943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b17ee5-b79d-4b9b-880a-ac231914c554 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:05.547234 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07187ff1-af08-4f7e-85a2-dcdd118cd24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:05.547984 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07187ff1-af08-4f7e-85a2-dcdd118cd24b admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:05.548361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07187ff1-af08-4f7e-85a2-dcdd118cd24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:05.548763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07187ff1-af08-4f7e-85a2-dcdd118cd24b admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:05.549492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07187ff1-af08-4f7e-85a2-dcdd118cd24b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:16:05.549835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07187ff1-af08-4f7e-85a2-dcdd118cd24b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:05.551777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07187ff1-af08-4f7e-85a2-dcdd118cd24b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:05.565694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1235/2467] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:16:05.570053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba052d-02f9-4a6d-861b-8a1755b0613c None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:05.583701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba052d-02f9-4a6d-861b-8a1755b0613c None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:05.590593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b28087-d998-4a56-b9b4-eb6c35fd02d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:05.591680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b28087-d998-4a56-b9b4-eb6c35fd02d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:05.592163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b28087-d998-4a56-b9b4-eb6c35fd02d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:05.593463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b28087-d998-4a56-b9b4-eb6c35fd02d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:05.594340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b28087-d998-4a56-b9b4-eb6c35fd02d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:05.598319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b28087-d998-4a56-b9b4-eb6c35fd02d7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:05.724825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1233/2468] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:05 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:05.729230 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07187ff1-af08-4f7e-85a2-dcdd118cd24b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:05.744882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07187ff1-af08-4f7e-85a2-dcdd118cd24b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:05.837428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a893897-625e-47b7-92b9-38d9cb3e1ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:05.838254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a893897-625e-47b7-92b9-38d9cb3e1ace admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:05.838805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a893897-625e-47b7-92b9-38d9cb3e1ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:05.839290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a893897-625e-47b7-92b9-38d9cb3e1ace admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:05.840159 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a893897-625e-47b7-92b9-38d9cb3e1ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:05.843295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a893897-625e-47b7-92b9-38d9cb3e1ace admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:05.927660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1236/2469] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:05 2022] POST /identity/v3/users => generated 284 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:05.931731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b28087-d998-4a56-b9b4-eb6c35fd02d7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:05.942636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b28087-d998-4a56-b9b4-eb6c35fd02d7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:05.957589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c644f53-6bf0-46bb-81cc-a13e2a983f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:05.958366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c644f53-6bf0-46bb-81cc-a13e2a983f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:05.958843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c644f53-6bf0-46bb-81cc-a13e2a983f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:05.960466 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c644f53-6bf0-46bb-81cc-a13e2a983f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:05.961305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c644f53-6bf0-46bb-81cc-a13e2a983f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:05.966216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c644f53-6bf0-46bb-81cc-a13e2a983f1a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:05.991453 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1234/2470] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:05.992447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a893897-625e-47b7-92b9-38d9cb3e1ace admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:06.023466 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a893897-625e-47b7-92b9-38d9cb3e1ace admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:06.056097 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3e226-331c-48c0-bcf2-a15890509eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:06.057011 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3e226-331c-48c0-bcf2-a15890509eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:06.057457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3e226-331c-48c0-bcf2-a15890509eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:06.057888 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3e226-331c-48c0-bcf2-a15890509eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:06.058740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3e226-331c-48c0-bcf2-a15890509eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:06.061804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3e226-331c-48c0-bcf2-a15890509eb1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:06.097265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1237/2471] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:06.098530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c644f53-6bf0-46bb-81cc-a13e2a983f1a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:06.114231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c644f53-6bf0-46bb-81cc-a13e2a983f1a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:06.130332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637e46e-6bfb-4bc2-9df4-ebd511a8ce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:06.132172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637e46e-6bfb-4bc2-9df4-ebd511a8ce26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:06.132658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637e46e-6bfb-4bc2-9df4-ebd511a8ce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:06.133736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637e46e-6bfb-4bc2-9df4-ebd511a8ce26 admin admin] PATH_INFO: `/v3/projects/73908dd67938474eaeb53ba6df46c678/users/7defc13d32424db8a14d641da279c47b/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:06.179570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637e46e-6bfb-4bc2-9df4-ebd511a8ce26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73908dd67938474eaeb53ba6df46c678, user_id=7defc13d32424db8a14d641da279c47b, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:06.182481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637e46e-6bfb-4bc2-9df4-ebd511a8ce26 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:06.315634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1235/2472] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:06 2022] PUT /identity/v3/projects/73908dd67938474eaeb53ba6df46c678/users/7defc13d32424db8a14d641da279c47b/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:06.316231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef19be-e907-49b9-b50e-9f39ce876b9d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:06.316612 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef19be-e907-49b9-b50e-9f39ce876b9d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:06.316888 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef19be-e907-49b9-b50e-9f39ce876b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:06.356639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:06.396340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-35ef19be-e907-49b9-b50e-9f39ce876b9d None None] MFA Rules not processed for user `7defc13d32424db8a14d641da279c47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:06.401339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-35ef19be-e907-49b9-b50e-9f39ce876b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:06.697919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1238/2473] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:06.698271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637e46e-6bfb-4bc2-9df4-ebd511a8ce26 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:06.830813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637e46e-6bfb-4bc2-9df4-ebd511a8ce26 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:06.835887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cee3b6-53b7-44d0-815a-c82770a81bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:06.839190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cee3b6-53b7-44d0-815a-c82770a81bfe admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:06.840056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cee3b6-53b7-44d0-815a-c82770a81bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:06.840399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cee3b6-53b7-44d0-815a-c82770a81bfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:06.841407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cee3b6-53b7-44d0-815a-c82770a81bfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:06.848847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cee3b6-53b7-44d0-815a-c82770a81bfe admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:06.974974 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1236/2474] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:06 2022] POST /identity/v3/projects => generated 367 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:06.977731 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef19be-e907-49b9-b50e-9f39ce876b9d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:06.993228 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef19be-e907-49b9-b50e-9f39ce876b9d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:07.176076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cec91b-6041-45bb-8526-0f3667363a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:07.176970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cec91b-6041-45bb-8526-0f3667363a53 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:07.177316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cec91b-6041-45bb-8526-0f3667363a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:07.177670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cec91b-6041-45bb-8526-0f3667363a53 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:07.178743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cec91b-6041-45bb-8526-0f3667363a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:07.180616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cec91b-6041-45bb-8526-0f3667363a53 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:07.290285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1239/2475] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:06 2022] POST /identity/v3/users => generated 290 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:07.296221 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cee3b6-53b7-44d0-815a-c82770a81bfe admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:07.348145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cee3b6-53b7-44d0-815a-c82770a81bfe admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:07.370473 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dd5b9-6ffd-4206-b855-64fd732b6dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:07.381456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dd5b9-6ffd-4206-b855-64fd732b6dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:07.381456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dd5b9-6ffd-4206-b855-64fd732b6dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:07.381456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dd5b9-6ffd-4206-b855-64fd732b6dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:07.381456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dd5b9-6ffd-4206-b855-64fd732b6dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:07.381456 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dd5b9-6ffd-4206-b855-64fd732b6dbc admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:07.399159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1237/2476] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:07.399484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cec91b-6041-45bb-8526-0f3667363a53 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:07.417262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cec91b-6041-45bb-8526-0f3667363a53 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:07.423473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e4169-73d5-412a-9484-ddd32f3eda01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:07.424411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e4169-73d5-412a-9484-ddd32f3eda01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:07.424967 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e4169-73d5-412a-9484-ddd32f3eda01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:07.425400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e4169-73d5-412a-9484-ddd32f3eda01 admin admin] PATH_INFO: `/v3/projects/820672f87b8f4c04896007caa4086c97/users/fbf669955c8947f0b8772f8bcd4f71f1/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:07.443752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e4169-73d5-412a-9484-ddd32f3eda01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820672f87b8f4c04896007caa4086c97, user_id=fbf669955c8947f0b8772f8bcd4f71f1, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:07.447577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e4169-73d5-412a-9484-ddd32f3eda01 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:07.513899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:07.583474 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dd5b9-6ffd-4206-b855-64fd732b6dbc admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:07.584263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1240/2477] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:07 2022] PUT /identity/v3/projects/820672f87b8f4c04896007caa4086c97/users/fbf669955c8947f0b8772f8bcd4f71f1/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:07.596994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dd5b9-6ffd-4206-b855-64fd732b6dbc admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:07.693979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b5e86-40ca-4d85-b549-ceb99854795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:07.694804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b5e86-40ca-4d85-b549-ceb99854795a admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:07.695101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b5e86-40ca-4d85-b549-ceb99854795a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:07.696018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b5e86-40ca-4d85-b549-ceb99854795a admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:07.696435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b5e86-40ca-4d85-b549-ceb99854795a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:07.698556 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b5e86-40ca-4d85-b549-ceb99854795a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:07.715557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1238/2478] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:07.719331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e4169-73d5-412a-9484-ddd32f3eda01 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:07.746503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e4169-73d5-412a-9484-ddd32f3eda01 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:07.754233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb43e4a-1ede-43ae-b106-f75b5757cdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:07.754908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb43e4a-1ede-43ae-b106-f75b5757cdfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:07.755379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb43e4a-1ede-43ae-b106-f75b5757cdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:07.755644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb43e4a-1ede-43ae-b106-f75b5757cdfe admin admin] PATH_INFO: `/v3/projects/820672f87b8f4c04896007caa4086c97/users/fbf669955c8947f0b8772f8bcd4f71f1/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:07.758214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb43e4a-1ede-43ae-b106-f75b5757cdfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=820672f87b8f4c04896007caa4086c97, user_id=fbf669955c8947f0b8772f8bcd4f71f1, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:07.761523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb43e4a-1ede-43ae-b106-f75b5757cdfe admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:07.790332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1241/2479] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:07 2022] PUT /identity/v3/projects/820672f87b8f4c04896007caa4086c97/users/fbf669955c8947f0b8772f8bcd4f71f1/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:07.799693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d383ddb-fc60-4153-8c94-d618868ab1db None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:07.800030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d383ddb-fc60-4153-8c94-d618868ab1db None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:07.800270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d383ddb-fc60-4153-8c94-d618868ab1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:07.870938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3d383ddb-fc60-4153-8c94-d618868ab1db None None] MFA Rules not processed for user `fbf669955c8947f0b8772f8bcd4f71f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:07.872932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3d383ddb-fc60-4153-8c94-d618868ab1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:07.984640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1239/2480] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:08.002496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb43e4a-1ede-43ae-b106-f75b5757cdfe admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:08.008832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bfb43e4a-1ede-43ae-b106-f75b5757cdfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:08.101069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb43e4a-1ede-43ae-b106-f75b5757cdfe admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:08.110326 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc92d2-43f4-442c-839d-b9ce445442ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:08.112965 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc92d2-43f4-442c-839d-b9ce445442ae service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:08.113241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc92d2-43f4-442c-839d-b9ce445442ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:08.115655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc92d2-43f4-442c-839d-b9ce445442ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:08.155748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc92d2-43f4-442c-839d-b9ce445442ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:08.157310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc92d2-43f4-442c-839d-b9ce445442ae service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:08.170322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1242/2481] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:08.332933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:09.579439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:11.789444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57566ad-fc88-4fa9-a4a1-7881f726cdbc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:11.790008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57566ad-fc88-4fa9-a4a1-7881f726cdbc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:11.790008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57566ad-fc88-4fa9-a4a1-7881f726cdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:11.832229 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a57566ad-fc88-4fa9-a4a1-7881f726cdbc None None] MFA Rules not processed for user `07fa6d8acbb44b5abac88a7f63fee601`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:11.833558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a57566ad-fc88-4fa9-a4a1-7881f726cdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:11.926467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1240/2482] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:11.934362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc92d2-43f4-442c-839d-b9ce445442ae service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:11.935818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-44bc92d2-43f4-442c-839d-b9ce445442ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:11.993441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57566ad-fc88-4fa9-a4a1-7881f726cdbc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:12.021274 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc92d2-43f4-442c-839d-b9ce445442ae service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:12.024539 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:12.030550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52ff8c-470c-423d-a654-f2ce426b02a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:12.031334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52ff8c-470c-423d-a654-f2ce426b02a8 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:12.031615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52ff8c-470c-423d-a654-f2ce426b02a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:12.031881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52ff8c-470c-423d-a654-f2ce426b02a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:12.034989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ea52ff8c-470c-423d-a654-f2ce426b02a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:12.075164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57566ad-fc88-4fa9-a4a1-7881f726cdbc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:12.081807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:12.082440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:12.082715 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:12.083011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] PATH_INFO: `/v3/users/7defc13d32424db8a14d641da279c47b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:12.084603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7defc13d32424db8a14d641da279c47b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:12.086562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:12.108497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52ff8c-470c-423d-a654-f2ce426b02a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:12.110467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52ff8c-470c-423d-a654-f2ce426b02a8 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:12.120722 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1243/2483] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:12.264177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7defc13d32424db8a14d641da279c47b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:12.327358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7defc13d32424db8a14d641da279c47b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:12.336150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7defc13d32424db8a14d641da279c47b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:12.337325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7defc13d32424db8a14d641da279c47b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:12.338043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7defc13d32424db8a14d641da279c47b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:12.362720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1241/2484] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:11 2022] DELETE /identity/v3/users/7defc13d32424db8a14d641da279c47b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:12.367210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fa108-bcec-4946-8d3e-d47713b9c036 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:12.367804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fa108-bcec-4946-8d3e-d47713b9c036 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:12.368150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fa108-bcec-4946-8d3e-d47713b9c036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:12.398743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d02fa108-bcec-4946-8d3e-d47713b9c036 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:12.402913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d02fa108-bcec-4946-8d3e-d47713b9c036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:12.475135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1244/2485] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:12.486961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:12.488350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:12.557150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc246c1b-79ee-40ce-8f45-6564b9221c0e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:12.563557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58633a73-704c-4c49-b2b3-10f3bb5d63c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:12.567330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633a73-704c-4c49-b2b3-10f3bb5d63c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:12.567330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633a73-704c-4c49-b2b3-10f3bb5d63c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:12.567330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633a73-704c-4c49-b2b3-10f3bb5d63c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:12.574684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58633a73-704c-4c49-b2b3-10f3bb5d63c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:12.576111 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58633a73-704c-4c49-b2b3-10f3bb5d63c2 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:12.586089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1242/2486] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:16:12.772614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fa108-bcec-4946-8d3e-d47713b9c036 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:12.785067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fa108-bcec-4946-8d3e-d47713b9c036 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:12.797451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84f7f4-09c6-4967-a0c1-8b0ec7d83c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:12.807400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84f7f4-09c6-4967-a0c1-8b0ec7d83c54 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:12.807400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84f7f4-09c6-4967-a0c1-8b0ec7d83c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:12.807400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84f7f4-09c6-4967-a0c1-8b0ec7d83c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:12.807400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2a84f7f4-09c6-4967-a0c1-8b0ec7d83c54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:12.819427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84f7f4-09c6-4967-a0c1-8b0ec7d83c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:12.821069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84f7f4-09c6-4967-a0c1-8b0ec7d83c54 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:12.835979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1245/2487] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:13.331626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58633a73-704c-4c49-b2b3-10f3bb5d63c2 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:13.334037 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-58633a73-704c-4c49-b2b3-10f3bb5d63c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:13.355154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58633a73-704c-4c49-b2b3-10f3bb5d63c2 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:13.360922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:13.362683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:13.363044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:13.363381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] PATH_INFO: `/v3/projects/73908dd67938474eaeb53ba6df46c678` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:13.365034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73908dd67938474eaeb53ba6df46c678)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:13.370065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:13.401013 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:13.487454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73908dd67938474eaeb53ba6df46c678', 'request_id': 'req-89b095f8-dabf-42f4-986c-81ac22ce01c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.492921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73908dd67938474eaeb53ba6df46c678', 'request_id': 'req-89b095f8-dabf-42f4-986c-81ac22ce01c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.572983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73908dd67938474eaeb53ba6df46c678', 'request_id': 'req-89b095f8-dabf-42f4-986c-81ac22ce01c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.586184 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] The token cache is being invalidate because project 73908dd67938474eaeb53ba6df46c678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:13.586761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.587413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.589489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1243/2488] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:13 2022] DELETE /identity/v3/projects/73908dd67938474eaeb53ba6df46c678 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:13.593408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84f7f4-09c6-4967-a0c1-8b0ec7d83c54 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:13.603530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84f7f4-09c6-4967-a0c1-8b0ec7d83c54 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:13.616437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:13.662941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:13.663566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:13.663841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:13.664108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] PATH_INFO: `/v3/users/fbf669955c8947f0b8772f8bcd4f71f1` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:13.665260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbf669955c8947f0b8772f8bcd4f71f1)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:13.667175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:13.845295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:13.847494 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:13.853547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbf669955c8947f0b8772f8bcd4f71f1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.867076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b095f8-dabf-42f4-986c-81ac22ce01c4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:13.872969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:13.874645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:13.875171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:13.875675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] PATH_INFO: `/v3/users/0f6e81dfe9bb4303b4bce9e6303104f9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:13.877133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6e81dfe9bb4303b4bce9e6303104f9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:13.879766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:13.938337 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbf669955c8947f0b8772f8bcd4f71f1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.946188 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbf669955c8947f0b8772f8bcd4f71f1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.947425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbf669955c8947f0b8772f8bcd4f71f1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.947556 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9485d14-45b9-4366-bcc2-08422c1f3cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbf669955c8947f0b8772f8bcd4f71f1'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:13.969292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1246/2489] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:13 2022] DELETE /identity/v3/users/fbf669955c8947f0b8772f8bcd4f71f1 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:14.110754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6e81dfe9bb4303b4bce9e6303104f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:14.209470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6e81dfe9bb4303b4bce9e6303104f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:14.224660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6e81dfe9bb4303b4bce9e6303104f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:14.225522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6e81dfe9bb4303b4bce9e6303104f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:14.226807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6e81dfe9bb4303b4bce9e6303104f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:14.278688 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48681e-3f77-444a-a740-341a6f839c91 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:14.278980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48681e-3f77-444a-a740-341a6f839c91 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:14.279359 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48681e-3f77-444a-a740-341a6f839c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:14.282926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1244/2490] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:13 2022] DELETE /identity/v3/users/0f6e81dfe9bb4303b4bce9e6303104f9 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:14.304049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7e48681e-3f77-444a-a740-341a6f839c91 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:14.305664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7e48681e-3f77-444a-a740-341a6f839c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:14.440680 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1247/2491] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:14.458789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:14.460282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:14.535660 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff41991-7a38-41bf-aec3-4fdb7f6723ac admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:14.541017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c3feb-9e5a-42c2-b15e-b343f1104910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:14.541834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c3feb-9e5a-42c2-b15e-b343f1104910 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:14.542306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c3feb-9e5a-42c2-b15e-b343f1104910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:14.542580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c3feb-9e5a-42c2-b15e-b343f1104910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:14.555043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c3feb-9e5a-42c2-b15e-b343f1104910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:14.557853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c3feb-9e5a-42c2-b15e-b343f1104910 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:14.572450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1245/2492] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:15.154563 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48681e-3f77-444a-a740-341a6f839c91 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:15.159923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7e48681e-3f77-444a-a740-341a6f839c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:15.195270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48681e-3f77-444a-a740-341a6f839c91 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:15.217870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:15.218793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:15.221198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:15.221653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] PATH_INFO: `/v3/projects/820672f87b8f4c04896007caa4086c97` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:15.226973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=820672f87b8f4c04896007caa4086c97)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:15.235477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:15.417639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:15.450331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820672f87b8f4c04896007caa4086c97', 'request_id': 'req-16a763a5-51df-4860-9df3-eb348826b288'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:15.464171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820672f87b8f4c04896007caa4086c97', 'request_id': 'req-16a763a5-51df-4860-9df3-eb348826b288'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:15.600224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820672f87b8f4c04896007caa4086c97', 'request_id': 'req-16a763a5-51df-4860-9df3-eb348826b288'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:15.653497 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] The token cache is being invalidate because project 820672f87b8f4c04896007caa4086c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:15.655372 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:15.666673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:15.674590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1248/2493] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:15 2022] DELETE /identity/v3/projects/820672f87b8f4c04896007caa4086c97 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:15.678638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dfaa1-7c68-46fd-b865-7e367ffdfe03 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:15.679098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dfaa1-7c68-46fd-b865-7e367ffdfe03 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:15.679098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dfaa1-7c68-46fd-b865-7e367ffdfe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:15.707303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-207dfaa1-7c68-46fd-b865-7e367ffdfe03 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:15.708949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-207dfaa1-7c68-46fd-b865-7e367ffdfe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:15.737232 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:15.769816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:15.776547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:15.810856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1246/2494] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:15.820477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95d4e3-5b19-4dfc-b6a4-1c52a8ff0b1f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:15.821130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95d4e3-5b19-4dfc-b6a4-1c52a8ff0b1f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:15.821529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95d4e3-5b19-4dfc-b6a4-1c52a8ff0b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:15.849596 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fb95d4e3-5b19-4dfc-b6a4-1c52a8ff0b1f None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:15.851961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fb95d4e3-5b19-4dfc-b6a4-1c52a8ff0b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:15.871788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1247/2495] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:15.875003 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95d4e3-5b19-4dfc-b6a4-1c52a8ff0b1f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:15.896703 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95d4e3-5b19-4dfc-b6a4-1c52a8ff0b1f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:15.908896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e16e-5e99-41a8-b5f9-034a36fdea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:15.909652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e16e-5e99-41a8-b5f9-034a36fdea3c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:15.910152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e16e-5e99-41a8-b5f9-034a36fdea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:15.910586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e16e-5e99-41a8-b5f9-034a36fdea3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:15.912274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e16e-5e99-41a8-b5f9-034a36fdea3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:16:15.913244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e16e-5e99-41a8-b5f9-034a36fdea3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:15.914992 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e16e-5e99-41a8-b5f9-034a36fdea3c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:15.945689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a763a5-51df-4860-9df3-eb348826b288 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:15.946243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1248/2496] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:16:15.948910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e16e-5e99-41a8-b5f9-034a36fdea3c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:15.961709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:15.961709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:15.962028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:15.962678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] PATH_INFO: `/v3/projects/d639f04384864de88d1cae7fdf2cb211` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:15.965189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d639f04384864de88d1cae7fdf2cb211)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:15.973142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:15.975354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e16e-5e99-41a8-b5f9-034a36fdea3c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:15.991230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a37b64-cedd-4f2d-b3c6-8d9bef2a818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:15.992197 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a37b64-cedd-4f2d-b3c6-8d9bef2a818c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:15.992736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a37b64-cedd-4f2d-b3c6-8d9bef2a818c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:15.993578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a37b64-cedd-4f2d-b3c6-8d9bef2a818c admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:15.996138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a37b64-cedd-4f2d-b3c6-8d9bef2a818c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:15.999147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a37b64-cedd-4f2d-b3c6-8d9bef2a818c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:16.159051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd639f04384864de88d1cae7fdf2cb211', 'request_id': 'req-05304a8d-d730-4de7-b298-1cedae713f76'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:16.172432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd639f04384864de88d1cae7fdf2cb211', 'request_id': 'req-05304a8d-d730-4de7-b298-1cedae713f76'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:16.203296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1249/2497] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:15 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:16.207780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a37b64-cedd-4f2d-b3c6-8d9bef2a818c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:16.224640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd639f04384864de88d1cae7fdf2cb211', 'request_id': 'req-05304a8d-d730-4de7-b298-1cedae713f76'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:16.254183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] The token cache is being invalidate because project d639f04384864de88d1cae7fdf2cb211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:16.254525 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:16.255438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-05304a8d-d730-4de7-b298-1cedae713f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:16.258153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1249/2498] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:15 2022] DELETE /identity/v3/projects/d639f04384864de88d1cae7fdf2cb211 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:16.262917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059fde8-4c7d-474d-b12c-4f34754a0238 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.263511 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059fde8-4c7d-474d-b12c-4f34754a0238 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.264994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059fde8-4c7d-474d-b12c-4f34754a0238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:16.294567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5059fde8-4c7d-474d-b12c-4f34754a0238 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:16.300709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5059fde8-4c7d-474d-b12c-4f34754a0238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:16.372365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a37b64-cedd-4f2d-b3c6-8d9bef2a818c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:16.399104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c81562-ed17-457b-926d-d36bf67d51db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:16.400766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c81562-ed17-457b-926d-d36bf67d51db admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.401291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c81562-ed17-457b-926d-d36bf67d51db admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.401789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c81562-ed17-457b-926d-d36bf67d51db admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:16.402705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c81562-ed17-457b-926d-d36bf67d51db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:16.405503 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c81562-ed17-457b-926d-d36bf67d51db admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:16.419682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1250/2499] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:16.428896 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8175c-d470-4d38-9480-1ef56fea18b4 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.429435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8175c-d470-4d38-9480-1ef56fea18b4 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.429987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8175c-d470-4d38-9480-1ef56fea18b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:16.473149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6fb8175c-d470-4d38-9480-1ef56fea18b4 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:16.474895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6fb8175c-d470-4d38-9480-1ef56fea18b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:16.486125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1251/2500] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:16.497937 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8175c-d470-4d38-9480-1ef56fea18b4 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:16.506459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1250/2501] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:16 2022] POST /identity/v3/users => generated 289 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:16.513106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8175c-d470-4d38-9480-1ef56fea18b4 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:16.515407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c81562-ed17-457b-926d-d36bf67d51db admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:16.521071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-95c81562-ed17-457b-926d-d36bf67d51db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:16.531849 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4f247-d343-45cc-8db5-825610ff5e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:16.533180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4f247-d343-45cc-8db5-825610ff5e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.533686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4f247-d343-45cc-8db5-825610ff5e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.534108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4f247-d343-45cc-8db5-825610ff5e12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:16.534982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4f247-d343-45cc-8db5-825610ff5e12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:16:16.535537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4f247-d343-45cc-8db5-825610ff5e12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:16.544844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c81562-ed17-457b-926d-d36bf67d51db admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:16.545855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4f247-d343-45cc-8db5-825610ff5e12 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:16.550016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5e300-ac8b-4506-8781-fa6f35f5852c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:16.550628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5e300-ac8b-4506-8781-fa6f35f5852c admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.550920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5e300-ac8b-4506-8781-fa6f35f5852c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.551239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5e300-ac8b-4506-8781-fa6f35f5852c admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:16.551816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5e300-ac8b-4506-8781-fa6f35f5852c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:16.553702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5e300-ac8b-4506-8781-fa6f35f5852c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:16.565151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1252/2502] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:16:16.571081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4f247-d343-45cc-8db5-825610ff5e12 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:16.590663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1251/2503] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:16.590858 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4f247-d343-45cc-8db5-825610ff5e12 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:16.592183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5e300-ac8b-4506-8781-fa6f35f5852c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:16.603546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5e300-ac8b-4506-8781-fa6f35f5852c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:16.606100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e545b5e-2d81-477b-8fff-70d8667f36b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:16.606599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e545b5e-2d81-477b-8fff-70d8667f36b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.608108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e545b5e-2d81-477b-8fff-70d8667f36b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.609174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd547a-f7e8-47c3-848c-ec66399b56b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:16.609308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e545b5e-2d81-477b-8fff-70d8667f36b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:16.609929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd547a-f7e8-47c3-848c-ec66399b56b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.610178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e545b5e-2d81-477b-8fff-70d8667f36b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:16.610296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd547a-f7e8-47c3-848c-ec66399b56b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.610674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd547a-f7e8-47c3-848c-ec66399b56b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:16.611388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd547a-f7e8-47c3-848c-ec66399b56b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:16.613335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd547a-f7e8-47c3-848c-ec66399b56b1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:16.614335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e545b5e-2d81-477b-8fff-70d8667f36b6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:16.649570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1252/2504] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:16.654427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd547a-f7e8-47c3-848c-ec66399b56b1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:16.675784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd547a-f7e8-47c3-848c-ec66399b56b1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:16.694662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d198e2-ab6c-49d5-b2af-a3f70dc88e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:16.695538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d198e2-ab6c-49d5-b2af-a3f70dc88e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.696056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d198e2-ab6c-49d5-b2af-a3f70dc88e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.696729 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d198e2-ab6c-49d5-b2af-a3f70dc88e49 admin admin] PATH_INFO: `/v3/projects/5e8e12824ef24876b78c154ae28a6b44/users/e078eedf2f094a1e86c4a23b700b04dd/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:16.717597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d198e2-ab6c-49d5-b2af-a3f70dc88e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8e12824ef24876b78c154ae28a6b44, user_id=e078eedf2f094a1e86c4a23b700b04dd, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:16.720844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d198e2-ab6c-49d5-b2af-a3f70dc88e49 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:16.724447 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1253/2505] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:16 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:16.725322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e545b5e-2d81-477b-8fff-70d8667f36b6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:16.738606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:16.828970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e545b5e-2d81-477b-8fff-70d8667f36b6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:16.835781 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1253/2506] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:16 2022] PUT /identity/v3/projects/5e8e12824ef24876b78c154ae28a6b44/users/e078eedf2f094a1e86c4a23b700b04dd/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:16.853579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb43a05-f5ca-4042-a59b-43376eb3f999 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.853867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb43a05-f5ca-4042-a59b-43376eb3f999 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.861569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb43a05-f5ca-4042-a59b-43376eb3f999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:16.925410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4cb43a05-f5ca-4042-a59b-43376eb3f999 None None] MFA Rules not processed for user `e078eedf2f094a1e86c4a23b700b04dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:16.927137 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4cb43a05-f5ca-4042-a59b-43376eb3f999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:16.979637 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d46b-44a7-4fc5-87a7-45088d5d4619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:16.982637 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d46b-44a7-4fc5-87a7-45088d5d4619 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:16.983482 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d46b-44a7-4fc5-87a7-45088d5d4619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:16.983766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d46b-44a7-4fc5-87a7-45088d5d4619 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.000920 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187d46b-44a7-4fc5-87a7-45088d5d4619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:17.005310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187d46b-44a7-4fc5-87a7-45088d5d4619 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:17.042113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:17.071274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1254/2507] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:17.077051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb43a05-f5ca-4042-a59b-43376eb3f999 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:17.096409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb43a05-f5ca-4042-a59b-43376eb3f999 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:17.122594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f106ec-0ac1-4c56-86c3-130bfa9c9664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:17.124585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f106ec-0ac1-4c56-86c3-130bfa9c9664 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:17.124976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f106ec-0ac1-4c56-86c3-130bfa9c9664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:17.125382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f106ec-0ac1-4c56-86c3-130bfa9c9664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.126175 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f106ec-0ac1-4c56-86c3-130bfa9c9664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:17.128666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f106ec-0ac1-4c56-86c3-130bfa9c9664 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:17.130974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1254/2508] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:16 2022] POST /identity/v3/users => generated 290 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:17.135208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d46b-44a7-4fc5-87a7-45088d5d4619 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:17.160439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d46b-44a7-4fc5-87a7-45088d5d4619 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:17.172045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44579e8b-64de-4d99-9854-a1d60453c583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:17.172930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44579e8b-64de-4d99-9854-a1d60453c583 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:17.173220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44579e8b-64de-4d99-9854-a1d60453c583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:17.173540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44579e8b-64de-4d99-9854-a1d60453c583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.174200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44579e8b-64de-4d99-9854-a1d60453c583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:17.175923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44579e8b-64de-4d99-9854-a1d60453c583 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:17.188720 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1255/2509] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:17.192473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44579e8b-64de-4d99-9854-a1d60453c583 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:17.209029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44579e8b-64de-4d99-9854-a1d60453c583 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:17.229142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234906f-34c1-4963-bdd4-643fd79c1c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:17.230109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234906f-34c1-4963-bdd4-643fd79c1c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:17.230546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234906f-34c1-4963-bdd4-643fd79c1c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:17.233521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234906f-34c1-4963-bdd4-643fd79c1c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.241873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234906f-34c1-4963-bdd4-643fd79c1c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:17.243830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234906f-34c1-4963-bdd4-643fd79c1c46 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:17.259754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1256/2510] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:17.266390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234906f-34c1-4963-bdd4-643fd79c1c46 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:17.301868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234906f-34c1-4963-bdd4-643fd79c1c46 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:17.322481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1255/2511] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:17 2022] POST /identity/v3/projects => generated 375 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:17.326432 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f106ec-0ac1-4c56-86c3-130bfa9c9664 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:17.327693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62566f77-6f0f-4e99-aa30-f0d785676a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:17.328380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62566f77-6f0f-4e99-aa30-f0d785676a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:17.328691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62566f77-6f0f-4e99-aa30-f0d785676a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:17.328965 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62566f77-6f0f-4e99-aa30-f0d785676a7d admin admin] PATH_INFO: `/v3/projects/d9ea80e3da8a4ceb9ef0b7011f7ea374/users/09cef90067284811a59c0691a3a12e58/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.354261 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62566f77-6f0f-4e99-aa30-f0d785676a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ea80e3da8a4ceb9ef0b7011f7ea374, user_id=09cef90067284811a59c0691a3a12e58, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:17.357505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62566f77-6f0f-4e99-aa30-f0d785676a7d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:17.417689 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f106ec-0ac1-4c56-86c3-130bfa9c9664 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:17.423553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfdf34-c2f5-4efa-b951-e5cf7deb6c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:17.424991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfdf34-c2f5-4efa-b951-e5cf7deb6c04 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:17.424991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfdf34-c2f5-4efa-b951-e5cf7deb6c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:17.424991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfdf34-c2f5-4efa-b951-e5cf7deb6c04 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.425645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfdf34-c2f5-4efa-b951-e5cf7deb6c04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:17.427677 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfdf34-c2f5-4efa-b951-e5cf7deb6c04 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:17.450510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1257/2512] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:17 2022] PUT /identity/v3/projects/d9ea80e3da8a4ceb9ef0b7011f7ea374/users/09cef90067284811a59c0691a3a12e58/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:17.461182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9dd4f-b965-4703-bc22-1bbd38a718f6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:17.461527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9dd4f-b965-4703-bc22-1bbd38a718f6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:17.461796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9dd4f-b965-4703-bc22-1bbd38a718f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.525206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-26b9dd4f-b965-4703-bc22-1bbd38a718f6 None None] MFA Rules not processed for user `09cef90067284811a59c0691a3a12e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:17.527088 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-26b9dd4f-b965-4703-bc22-1bbd38a718f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:17.544325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1256/2513] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:17 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:17.545930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfdf34-c2f5-4efa-b951-e5cf7deb6c04 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:17.559490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfdf34-c2f5-4efa-b951-e5cf7deb6c04 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:17.689366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1258/2514] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:17.694810 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9dd4f-b965-4703-bc22-1bbd38a718f6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:17.697318 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d1587-2e48-40ce-be9e-da05a1b659b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:17.698533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d1587-2e48-40ce-be9e-da05a1b659b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:17.699025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d1587-2e48-40ce-be9e-da05a1b659b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:17.699507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d1587-2e48-40ce-be9e-da05a1b659b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.700339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d1587-2e48-40ce-be9e-da05a1b659b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:17.707254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d1587-2e48-40ce-be9e-da05a1b659b3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:17.711489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9dd4f-b965-4703-bc22-1bbd38a718f6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:17.718557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1257/2515] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:17.722454 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d1587-2e48-40ce-be9e-da05a1b659b3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:17.727606 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3a6d2-0731-473f-99b7-11e7df0b5756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:17.729437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3a6d2-0731-473f-99b7-11e7df0b5756 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:17.730376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3a6d2-0731-473f-99b7-11e7df0b5756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:17.730503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3a6d2-0731-473f-99b7-11e7df0b5756 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.731227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3a6d2-0731-473f-99b7-11e7df0b5756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:17.735131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3a6d2-0731-473f-99b7-11e7df0b5756 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:17.741661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d1587-2e48-40ce-be9e-da05a1b659b3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:17.764717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c67d4-fbe9-4dba-ab24-f9da55631e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:17.772110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c67d4-fbe9-4dba-ab24-f9da55631e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:17.772110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c67d4-fbe9-4dba-ab24-f9da55631e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:17.772110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c67d4-fbe9-4dba-ab24-f9da55631e95 admin admin] PATH_INFO: `/v3/projects/758b38ad039445c7a4dc502283977c0e/users/87efa71f25d8411098ccd3088f584f4e/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:17.800095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c67d4-fbe9-4dba-ab24-f9da55631e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758b38ad039445c7a4dc502283977c0e, user_id=87efa71f25d8411098ccd3088f584f4e, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:17.800882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c67d4-fbe9-4dba-ab24-f9da55631e95 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:17.906502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1259/2516] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:17 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:17.914125 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3a6d2-0731-473f-99b7-11e7df0b5756 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:17.923303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1258/2517] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:17 2022] PUT /identity/v3/projects/758b38ad039445c7a4dc502283977c0e/users/87efa71f25d8411098ccd3088f584f4e/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:17.926086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c67d4-fbe9-4dba-ab24-f9da55631e95 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:18.051877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c67d4-fbe9-4dba-ab24-f9da55631e95 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:18.082922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3a6d2-0731-473f-99b7-11e7df0b5756 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:18.100573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93bda2-0079-4076-88b2-e41f06aa18cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:18.101333 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93bda2-0079-4076-88b2-e41f06aa18cd admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.101735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93bda2-0079-4076-88b2-e41f06aa18cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.102102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93bda2-0079-4076-88b2-e41f06aa18cd admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:18.102931 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93bda2-0079-4076-88b2-e41f06aa18cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:18.104973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93bda2-0079-4076-88b2-e41f06aa18cd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:18.128808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37fc9f-43d6-44c2-9152-6f9275dfa990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:18.129855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37fc9f-43d6-44c2-9152-6f9275dfa990 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.130329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37fc9f-43d6-44c2-9152-6f9275dfa990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.130945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37fc9f-43d6-44c2-9152-6f9275dfa990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:18.131835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37fc9f-43d6-44c2-9152-6f9275dfa990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:18.133936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37fc9f-43d6-44c2-9152-6f9275dfa990 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:18.149523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1259/2518] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:18.151827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37fc9f-43d6-44c2-9152-6f9275dfa990 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:18.169597 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37fc9f-43d6-44c2-9152-6f9275dfa990 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:18.177236 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06060a96-88d7-442a-ac58-43e0a287537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:18.177998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06060a96-88d7-442a-ac58-43e0a287537e admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.178401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06060a96-88d7-442a-ac58-43e0a287537e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.178756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06060a96-88d7-442a-ac58-43e0a287537e admin admin] PATH_INFO: `/v3/projects/758b38ad039445c7a4dc502283977c0e/users/87efa71f25d8411098ccd3088f584f4e/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:18.180941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06060a96-88d7-442a-ac58-43e0a287537e admin admin] RBAC: Authorizing `identity:create_grant(project_id=758b38ad039445c7a4dc502283977c0e, user_id=87efa71f25d8411098ccd3088f584f4e, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:18.184212 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06060a96-88d7-442a-ac58-43e0a287537e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:18.200588 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1260/2519] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:18 2022] PUT /identity/v3/projects/758b38ad039445c7a4dc502283977c0e/users/87efa71f25d8411098ccd3088f584f4e/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:18.223741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8cb37-faa4-43a3-bfc8-b3b8d0696426 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.223741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8cb37-faa4-43a3-bfc8-b3b8d0696426 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.223741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8cb37-faa4-43a3-bfc8-b3b8d0696426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:18.248095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1260/2520] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:17 2022] POST /identity/v3/users => generated 297 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:18.250212 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93bda2-0079-4076-88b2-e41f06aa18cd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:18.263883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93bda2-0079-4076-88b2-e41f06aa18cd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:18.298950 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7cf8cb37-faa4-43a3-bfc8-b3b8d0696426 None None] MFA Rules not processed for user `87efa71f25d8411098ccd3088f584f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:18.300162 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7cf8cb37-faa4-43a3-bfc8-b3b8d0696426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:18.304326 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:18.360469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02515283-6863-4f7c-8e1c-bb3e6b30ca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:18.361315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02515283-6863-4f7c-8e1c-bb3e6b30ca1a admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.361698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02515283-6863-4f7c-8e1c-bb3e6b30ca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.362326 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02515283-6863-4f7c-8e1c-bb3e6b30ca1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:18.363191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02515283-6863-4f7c-8e1c-bb3e6b30ca1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:18.364945 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02515283-6863-4f7c-8e1c-bb3e6b30ca1a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:18.382069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1261/2521] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:18.383156 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02515283-6863-4f7c-8e1c-bb3e6b30ca1a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:18.395187 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02515283-6863-4f7c-8e1c-bb3e6b30ca1a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:18.401386 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e8c6a-299b-4473-8111-f0305f7a760b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:18.402179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e8c6a-299b-4473-8111-f0305f7a760b admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.402554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e8c6a-299b-4473-8111-f0305f7a760b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.402922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e8c6a-299b-4473-8111-f0305f7a760b admin admin] PATH_INFO: `/v3/projects/79b9691590c3490f94757b7055714ea1/users/4111366e46444f188098079090808841/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:18.420072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:18.459498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1261/2522] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:18.469277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e8c6a-299b-4473-8111-f0305f7a760b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b9691590c3490f94757b7055714ea1, user_id=4111366e46444f188098079090808841, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:18.476759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8cb37-faa4-43a3-bfc8-b3b8d0696426 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:18.478283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7cf8cb37-faa4-43a3-bfc8-b3b8d0696426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:18.481999 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e8c6a-299b-4473-8111-f0305f7a760b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:18.577329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8cb37-faa4-43a3-bfc8-b3b8d0696426 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:18.581426 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db169764-ef89-4b99-9af7-3fb90b005084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:18.582033 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db169764-ef89-4b99-9af7-3fb90b005084 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.582505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db169764-ef89-4b99-9af7-3fb90b005084 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.582859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db169764-ef89-4b99-9af7-3fb90b005084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:18.590179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1262/2523] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:18 2022] PUT /identity/v3/projects/79b9691590c3490f94757b7055714ea1/users/4111366e46444f188098079090808841/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:18.608185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e8c6a-299b-4473-8111-f0305f7a760b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:18.742861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db169764-ef89-4b99-9af7-3fb90b005084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:18.746569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db169764-ef89-4b99-9af7-3fb90b005084 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:18.780968 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e8c6a-299b-4473-8111-f0305f7a760b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:18.784658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1262/2524] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:18.819488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449c0e9-52e1-41ae-9c71-f34cf4589564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:18.820127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c0e9-52e1-41ae-9c71-f34cf4589564 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.820407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c0e9-52e1-41ae-9c71-f34cf4589564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.821541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c0e9-52e1-41ae-9c71-f34cf4589564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:18.821919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449c0e9-52e1-41ae-9c71-f34cf4589564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:18.829065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449c0e9-52e1-41ae-9c71-f34cf4589564 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:18.852734 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db169764-ef89-4b99-9af7-3fb90b005084 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:18.867635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:18.868154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1263/2525] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:18.872538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db169764-ef89-4b99-9af7-3fb90b005084 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:18.885849 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd60c-4869-4f35-9437-0d89df083553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:18.886541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd60c-4869-4f35-9437-0d89df083553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.886819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd60c-4869-4f35-9437-0d89df083553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.890348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd60c-4869-4f35-9437-0d89df083553 admin admin] PATH_INFO: `/v3/projects/79b9691590c3490f94757b7055714ea1/users/4111366e46444f188098079090808841/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:18.898786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:18.904341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecd60c-4869-4f35-9437-0d89df083553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b9691590c3490f94757b7055714ea1, user_id=4111366e46444f188098079090808841, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:18.915891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecd60c-4869-4f35-9437-0d89df083553 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:18.951069 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1263/2526] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:18 2022] PUT /identity/v3/projects/79b9691590c3490f94757b7055714ea1/users/4111366e46444f188098079090808841/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:18.965272 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fea3b1-8f97-48a2-89d2-60c0e306a00d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:18.965673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fea3b1-8f97-48a2-89d2-60c0e306a00d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:18.965811 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fea3b1-8f97-48a2-89d2-60c0e306a00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:19.049119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-15fea3b1-8f97-48a2-89d2-60c0e306a00d None None] MFA Rules not processed for user `4111366e46444f188098079090808841`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:19.049119 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-15fea3b1-8f97-48a2-89d2-60c0e306a00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:19.213238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1264/2527] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:19.227536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd60c-4869-4f35-9437-0d89df083553 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:19.229417 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1aecd60c-4869-4f35-9437-0d89df083553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:19.363081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd60c-4869-4f35-9437-0d89df083553 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:19.372639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dba641-12fe-4e42-b42c-390f6afdbb1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:19.373964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dba641-12fe-4e42-b42c-390f6afdbb1e service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:19.374410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dba641-12fe-4e42-b42c-390f6afdbb1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:19.374787 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dba641-12fe-4e42-b42c-390f6afdbb1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:19.401105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dba641-12fe-4e42-b42c-390f6afdbb1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:19.403373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dba641-12fe-4e42-b42c-390f6afdbb1e service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:19.419072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1264/2528] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:19.928073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fea3b1-8f97-48a2-89d2-60c0e306a00d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:19.941522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fea3b1-8f97-48a2-89d2-60c0e306a00d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:19.958959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bfde2-a7ef-450e-b2ff-902c92bd965f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:19.958959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bfde2-a7ef-450e-b2ff-902c92bd965f service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:19.958959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bfde2-a7ef-450e-b2ff-902c92bd965f service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:19.958959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bfde2-a7ef-450e-b2ff-902c92bd965f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:20.023381 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:20.032692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bfde2-a7ef-450e-b2ff-902c92bd965f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:20.034128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bfde2-a7ef-450e-b2ff-902c92bd965f service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:20.043317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1265/2529] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:22.692191 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dba641-12fe-4e42-b42c-390f6afdbb1e service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:22.693970 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-f0dba641-12fe-4e42-b42c-390f6afdbb1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:22.756129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dba641-12fe-4e42-b42c-390f6afdbb1e service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:22.759779 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fae488-27ef-4075-8398-e211580d3a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:22.760380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fae488-27ef-4075-8398-e211580d3a3f service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:22.760692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fae488-27ef-4075-8398-e211580d3a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:22.760961 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fae488-27ef-4075-8398-e211580d3a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:22.762670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-14fae488-27ef-4075-8398-e211580d3a3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:22.771905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:22.854329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fae488-27ef-4075-8398-e211580d3a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:22.855777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fae488-27ef-4075-8398-e211580d3a3f service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:22.869490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1265/2530] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:23.115105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bfde2-a7ef-450e-b2ff-902c92bd965f service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:23.117917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f29bfde2-a7ef-450e-b2ff-902c92bd965f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:23.137175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bfde2-a7ef-450e-b2ff-902c92bd965f service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:23.140581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:23.141288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:23.141567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:23.141980 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] PATH_INFO: `/v3/users/b7d7f6721a3a40f380578391949c443b` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:23.143089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d7f6721a3a40f380578391949c443b)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:23.145018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:23.352117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d7f6721a3a40f380578391949c443b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:23.441169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d7f6721a3a40f380578391949c443b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:23.450733 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d7f6721a3a40f380578391949c443b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:23.452411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d7f6721a3a40f380578391949c443b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:23.453650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d7f6721a3a40f380578391949c443b'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:23.475219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1266/2531] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:23 2022] DELETE /identity/v3/users/b7d7f6721a3a40f380578391949c443b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:23.785767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fae488-27ef-4075-8398-e211580d3a3f service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:23.798018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fae488-27ef-4075-8398-e211580d3a3f service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:23.870061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:23.870769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:23.871047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:23.871314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] PATH_INFO: `/v3/projects/64273458329d44edb0e2904b4011b591` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:23.872855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64273458329d44edb0e2904b4011b591)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:23.878803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:24.017176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64273458329d44edb0e2904b4011b591', 'request_id': 'req-440703a6-ff92-4ca3-9c6e-d70d45b32527'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.023887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64273458329d44edb0e2904b4011b591', 'request_id': 'req-440703a6-ff92-4ca3-9c6e-d70d45b32527'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.098566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64273458329d44edb0e2904b4011b591', 'request_id': 'req-440703a6-ff92-4ca3-9c6e-d70d45b32527'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.111625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] The token cache is being invalidate because project 64273458329d44edb0e2904b4011b591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:24.111916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.112587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.114296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1266/2532] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:23 2022] DELETE /identity/v3/projects/64273458329d44edb0e2904b4011b591 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:24.118308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:24.119802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:24.132755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28ccbe-0105-4429-9607-dbd88305219a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:24.190256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:24.190875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:24.191199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:24.191485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] PATH_INFO: `/v3/users/eee2e0d3e78f41ec83e2905623c46f1c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:24.192701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee2e0d3e78f41ec83e2905623c46f1c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:24.194604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:24.397553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee2e0d3e78f41ec83e2905623c46f1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.520988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee2e0d3e78f41ec83e2905623c46f1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.528674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee2e0d3e78f41ec83e2905623c46f1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.529338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee2e0d3e78f41ec83e2905623c46f1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.529973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee2e0d3e78f41ec83e2905623c46f1c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:24.532053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:24.550287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1267/2533] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:24 2022] DELETE /identity/v3/users/eee2e0d3e78f41ec83e2905623c46f1c => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:24.942217 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:24.943675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:24.957098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440703a6-ff92-4ca3-9c6e-d70d45b32527 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:24.998188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:25.022978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:25.023615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:25.023897 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:25.024164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] PATH_INFO: `/v3/projects/8d3fe8dee5294dddba8a43e1e3a4a45f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:25.025330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d3fe8dee5294dddba8a43e1e3a4a45f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:25.027242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:25.263057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d3fe8dee5294dddba8a43e1e3a4a45f', 'request_id': 'req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:25.272449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d3fe8dee5294dddba8a43e1e3a4a45f', 'request_id': 'req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:25.355865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d3fe8dee5294dddba8a43e1e3a4a45f', 'request_id': 'req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:25.368727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] The token cache is being invalidate because project 8d3fe8dee5294dddba8a43e1e3a4a45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:25.369007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:25.369638 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:25.371193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1267/2534] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:24 2022] DELETE /identity/v3/projects/8d3fe8dee5294dddba8a43e1e3a4a45f => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:25.982450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:25.983957 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:26.045242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae1265-d0e8-4625-8fbc-cafadb0509ae admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:26.097042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ee43c-8bdb-43f7-9c25-51e79f469f11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:26.097670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ee43c-8bdb-43f7-9c25-51e79f469f11 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:26.100086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ee43c-8bdb-43f7-9c25-51e79f469f11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:26.100353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ee43c-8bdb-43f7-9c25-51e79f469f11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:26.103225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-560ee43c-8bdb-43f7-9c25-51e79f469f11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:26.207783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ee43c-8bdb-43f7-9c25-51e79f469f11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:26.209316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ee43c-8bdb-43f7-9c25-51e79f469f11 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:26.221601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1268/2535] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:26.974089 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:26.975598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:27.061134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dfa77-c9be-48f5-a36c-c7f1fa2fd3bb admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:27.066306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:27.067151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:27.067566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:27.067909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] PATH_INFO: `/v3/users/2ec53f935a6c41f2af67e9fd2e13e12c` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:27.072047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec53f935a6c41f2af67e9fd2e13e12c)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:27.075064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:27.296139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec53f935a6c41f2af67e9fd2e13e12c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.365817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec53f935a6c41f2af67e9fd2e13e12c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.373550 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec53f935a6c41f2af67e9fd2e13e12c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.374341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec53f935a6c41f2af67e9fd2e13e12c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.374948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-15dd784e-1d96-4e56-89bc-9801258ba2f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec53f935a6c41f2af67e9fd2e13e12c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.379978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:27.393420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1268/2536] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:26 2022] DELETE /identity/v3/users/2ec53f935a6c41f2af67e9fd2e13e12c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:27.677930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ee43c-8bdb-43f7-9c25-51e79f469f11 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:27.691401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ee43c-8bdb-43f7-9c25-51e79f469f11 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:27.740861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:27.754037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:27.754698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:27.754941 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:27.755203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] PATH_INFO: `/v3/projects/cf197791c8734703b20784498cd8dcd5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:27.756292 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf197791c8734703b20784498cd8dcd5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:27.758223 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:27.859254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf197791c8734703b20784498cd8dcd5', 'request_id': 'req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.864790 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf197791c8734703b20784498cd8dcd5', 'request_id': 'req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.932379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf197791c8734703b20784498cd8dcd5', 'request_id': 'req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.944100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] The token cache is being invalidate because project cf197791c8734703b20784498cd8dcd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:27.944610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.945678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-7dd91afd-38d2-4a99-87e3-f3cbe882c0bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:27.947219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1269/2537] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:27 2022] DELETE /identity/v3/projects/cf197791c8734703b20784498cd8dcd5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:27.952398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f06ab-08a1-4855-bb64-cc08bc1e32f5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:27.957164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f06ab-08a1-4855-bb64-cc08bc1e32f5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:27.957164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f06ab-08a1-4855-bb64-cc08bc1e32f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:27.973575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-880f06ab-08a1-4855-bb64-cc08bc1e32f5 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:27.974788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-880f06ab-08a1-4855-bb64-cc08bc1e32f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:28.036927 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1269/2538] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:28.045889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da263750-dc1c-4fd4-b25d-923b1c0839e3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:28.046175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da263750-dc1c-4fd4-b25d-923b1c0839e3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:28.046446 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da263750-dc1c-4fd4-b25d-923b1c0839e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:28.067596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-da263750-dc1c-4fd4-b25d-923b1c0839e3 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:28.068862 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-da263750-dc1c-4fd4-b25d-923b1c0839e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:28.076988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1270/2539] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:28.081458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f06ab-08a1-4855-bb64-cc08bc1e32f5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:28.094619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f06ab-08a1-4855-bb64-cc08bc1e32f5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:28.103935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24440ef5-45da-46df-b9fa-cc0f88af8d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:28.104760 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24440ef5-45da-46df-b9fa-cc0f88af8d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:28.104989 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24440ef5-45da-46df-b9fa-cc0f88af8d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:28.105102 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24440ef5-45da-46df-b9fa-cc0f88af8d46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:28.105783 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24440ef5-45da-46df-b9fa-cc0f88af8d46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:16:28.106110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24440ef5-45da-46df-b9fa-cc0f88af8d46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:28.107722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24440ef5-45da-46df-b9fa-cc0f88af8d46 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:28.120844 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1270/2540] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:16:28.125569 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da263750-dc1c-4fd4-b25d-923b1c0839e3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:28.135120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da263750-dc1c-4fd4-b25d-923b1c0839e3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:28.140799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43bfba-d7fa-4952-a433-f46bca1ecdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:28.141366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43bfba-d7fa-4952-a433-f46bca1ecdf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:28.141635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43bfba-d7fa-4952-a433-f46bca1ecdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:28.141897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43bfba-d7fa-4952-a433-f46bca1ecdf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:28.142509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43bfba-d7fa-4952-a433-f46bca1ecdf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:28.144398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43bfba-d7fa-4952-a433-f46bca1ecdf0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:28.340649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1271/2541] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:28 2022] POST /identity/v3/projects => generated 375 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:28.345483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24440ef5-45da-46df-b9fa-cc0f88af8d46 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:28.355788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24440ef5-45da-46df-b9fa-cc0f88af8d46 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:28.413176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb39f2-ed9a-4a79-aecd-2aa20b8a8bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:28.413786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb39f2-ed9a-4a79-aecd-2aa20b8a8bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:28.414065 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb39f2-ed9a-4a79-aecd-2aa20b8a8bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:28.414336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb39f2-ed9a-4a79-aecd-2aa20b8a8bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:28.414993 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb39f2-ed9a-4a79-aecd-2aa20b8a8bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:28.417047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb39f2-ed9a-4a79-aecd-2aa20b8a8bb7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:28.499088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1271/2542] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:28 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:28.503472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43bfba-d7fa-4952-a433-f46bca1ecdf0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:28.514892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43bfba-d7fa-4952-a433-f46bca1ecdf0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:28.520181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cc202-bf05-4f73-b498-5929c9dfd315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:28.520859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cc202-bf05-4f73-b498-5929c9dfd315 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:28.521084 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cc202-bf05-4f73-b498-5929c9dfd315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:28.521350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cc202-bf05-4f73-b498-5929c9dfd315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:28.521994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cc202-bf05-4f73-b498-5929c9dfd315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:28.523609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cc202-bf05-4f73-b498-5929c9dfd315 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:28.534823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1272/2543] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:28.538710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb39f2-ed9a-4a79-aecd-2aa20b8a8bb7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:28.548784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb39f2-ed9a-4a79-aecd-2aa20b8a8bb7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:28.553629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08d886-28ab-41c8-9364-f8905922a8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:28.554225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08d886-28ab-41c8-9364-f8905922a8bb admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:28.554504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08d886-28ab-41c8-9364-f8905922a8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:28.554802 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08d886-28ab-41c8-9364-f8905922a8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:28.555441 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08d886-28ab-41c8-9364-f8905922a8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:28.557103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08d886-28ab-41c8-9364-f8905922a8bb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:28.568155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1272/2544] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:28.572263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cc202-bf05-4f73-b498-5929c9dfd315 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:28.583527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cc202-bf05-4f73-b498-5929c9dfd315 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:28.590791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2d176-eeac-4086-8f73-992193840899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:28.591449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2d176-eeac-4086-8f73-992193840899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:28.591724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2d176-eeac-4086-8f73-992193840899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:28.591992 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2d176-eeac-4086-8f73-992193840899 admin admin] PATH_INFO: `/v3/projects/260afaf425fe4faf980b4e66806c7016/users/220214173b5d4c5cac74e256a4b3a549/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:28.606769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2d176-eeac-4086-8f73-992193840899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260afaf425fe4faf980b4e66806c7016, user_id=220214173b5d4c5cac74e256a4b3a549, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:28.609521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2d176-eeac-4086-8f73-992193840899 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:28.687273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1273/2545] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:28 2022] PUT /identity/v3/projects/260afaf425fe4faf980b4e66806c7016/users/220214173b5d4c5cac74e256a4b3a549/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:28.699412 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69640056-26ea-4818-a972-7dd310e0eebe None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:28.699694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69640056-26ea-4818-a972-7dd310e0eebe None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:28.700079 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69640056-26ea-4818-a972-7dd310e0eebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:28.729158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-69640056-26ea-4818-a972-7dd310e0eebe None None] MFA Rules not processed for user `220214173b5d4c5cac74e256a4b3a549`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:28.730520 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-69640056-26ea-4818-a972-7dd310e0eebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:28.773725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:28.798299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1273/2546] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:28.810306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2d176-eeac-4086-8f73-992193840899 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:28.812092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-19b2d176-eeac-4086-8f73-992193840899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:28.890102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2d176-eeac-4086-8f73-992193840899 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:28.896728 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29114d36-65b9-4cf0-81ac-df15e86742a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:28.897348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29114d36-65b9-4cf0-81ac-df15e86742a2 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:28.897628 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29114d36-65b9-4cf0-81ac-df15e86742a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:28.897908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29114d36-65b9-4cf0-81ac-df15e86742a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:28.906614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29114d36-65b9-4cf0-81ac-df15e86742a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:28.908928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29114d36-65b9-4cf0-81ac-df15e86742a2 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:28.912426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:28.918143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1274/2547] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:29.150330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69640056-26ea-4818-a972-7dd310e0eebe None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:29.220806 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69640056-26ea-4818-a972-7dd310e0eebe None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:29.225809 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:29.226711 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:29.227066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:29.227362 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] PATH_INFO: `/v3/users/220214173b5d4c5cac74e256a4b3a549` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:29.228624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=220214173b5d4c5cac74e256a4b3a549)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:29.230788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:29.488530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '220214173b5d4c5cac74e256a4b3a549'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:29.549724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '220214173b5d4c5cac74e256a4b3a549'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:29.557612 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '220214173b5d4c5cac74e256a4b3a549'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:29.558347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '220214173b5d4c5cac74e256a4b3a549'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:29.558960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '220214173b5d4c5cac74e256a4b3a549'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:29.576663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1274/2548] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:29 2022] DELETE /identity/v3/users/220214173b5d4c5cac74e256a4b3a549 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:29.582009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1af5-3fc2-437a-a288-af2a3267af08 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:29.582297 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1af5-3fc2-437a-a288-af2a3267af08 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:29.582576 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1af5-3fc2-437a-a288-af2a3267af08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:29.601173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-106c1af5-3fc2-437a-a288-af2a3267af08 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:29.602378 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-106c1af5-3fc2-437a-a288-af2a3267af08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:29.661768 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1275/2549] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:29.672291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:29.677765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:29.730555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea5f16-1a95-4194-b990-a530032e2974 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:29.734081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da7ac1-faed-406e-ae50-3a3dca5765c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:29.734810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da7ac1-faed-406e-ae50-3a3dca5765c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:29.735429 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da7ac1-faed-406e-ae50-3a3dca5765c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:29.735767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da7ac1-faed-406e-ae50-3a3dca5765c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:29.745004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da7ac1-faed-406e-ae50-3a3dca5765c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:29.746421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da7ac1-faed-406e-ae50-3a3dca5765c8 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:29.756800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1275/2550] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:30.198930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1af5-3fc2-437a-a288-af2a3267af08 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:30.203469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-106c1af5-3fc2-437a-a288-af2a3267af08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:30.217263 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1af5-3fc2-437a-a288-af2a3267af08 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:30.222761 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:30.223468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:30.223817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:30.224676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] PATH_INFO: `/v3/projects/260afaf425fe4faf980b4e66806c7016` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:30.226095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=260afaf425fe4faf980b4e66806c7016)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:30.228345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:30.368566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260afaf425fe4faf980b4e66806c7016', 'request_id': 'req-1680ce77-a72e-4621-bef5-3271dba07f56'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:30.374152 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260afaf425fe4faf980b4e66806c7016', 'request_id': 'req-1680ce77-a72e-4621-bef5-3271dba07f56'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:30.486400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260afaf425fe4faf980b4e66806c7016', 'request_id': 'req-1680ce77-a72e-4621-bef5-3271dba07f56'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:30.497950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] The token cache is being invalidate because project 260afaf425fe4faf980b4e66806c7016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:30.498232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:30.498874 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1680ce77-a72e-4621-bef5-3271dba07f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:30.500339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1276/2551] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:30 2022] DELETE /identity/v3/projects/260afaf425fe4faf980b4e66806c7016 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:30.505301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce15ed-33ad-4ad5-a49b-490690921c1d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:30.505795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce15ed-33ad-4ad5-a49b-490690921c1d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:30.505900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce15ed-33ad-4ad5-a49b-490690921c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:30.513396 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:30.526142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-13ce15ed-33ad-4ad5-a49b-490690921c1d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:30.527335 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-13ce15ed-33ad-4ad5-a49b-490690921c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:30.585048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1276/2552] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:30.595158 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f132-3dfb-4f47-b32b-f20219ff8534 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:30.595432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f132-3dfb-4f47-b32b-f20219ff8534 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:30.595696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f132-3dfb-4f47-b32b-f20219ff8534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:30.615128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d7e3f132-3dfb-4f47-b32b-f20219ff8534 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:30.618648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d7e3f132-3dfb-4f47-b32b-f20219ff8534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:30.624268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1277/2553] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:30.628300 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce15ed-33ad-4ad5-a49b-490690921c1d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:30.638097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce15ed-33ad-4ad5-a49b-490690921c1d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:30.643189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fa92e-6d00-4c7e-abce-84bf3d8c105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:30.643776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fa92e-6d00-4c7e-abce-84bf3d8c105d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:30.644051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fa92e-6d00-4c7e-abce-84bf3d8c105d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:30.644322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fa92e-6d00-4c7e-abce-84bf3d8c105d admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:30.644994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fa92e-6d00-4c7e-abce-84bf3d8c105d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:16:30.645291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fa92e-6d00-4c7e-abce-84bf3d8c105d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:30.646892 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fa92e-6d00-4c7e-abce-84bf3d8c105d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:30.658882 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1277/2554] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:16:30.662864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3f132-3dfb-4f47-b32b-f20219ff8534 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:30.671946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3f132-3dfb-4f47-b32b-f20219ff8534 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:30.676613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5252965-5214-42df-b2b4-7d4f029d9149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:30.677196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5252965-5214-42df-b2b4-7d4f029d9149 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:30.677470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5252965-5214-42df-b2b4-7d4f029d9149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:30.677741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5252965-5214-42df-b2b4-7d4f029d9149 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:30.678364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5252965-5214-42df-b2b4-7d4f029d9149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:30.680247 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5252965-5214-42df-b2b4-7d4f029d9149 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:30.778321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1278/2555] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:30 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:30.782475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fa92e-6d00-4c7e-abce-84bf3d8c105d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:30.794274 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fa92e-6d00-4c7e-abce-84bf3d8c105d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:30.855720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c5609-9664-43bd-8db5-4467f9fc8a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:30.856323 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c5609-9664-43bd-8db5-4467f9fc8a7a admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:30.856628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c5609-9664-43bd-8db5-4467f9fc8a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:30.856896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c5609-9664-43bd-8db5-4467f9fc8a7a admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:30.857521 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c5609-9664-43bd-8db5-4467f9fc8a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:30.859444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c5609-9664-43bd-8db5-4467f9fc8a7a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:30.973459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1278/2556] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:30 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:30.977047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5252965-5214-42df-b2b4-7d4f029d9149 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:30.994416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5252965-5214-42df-b2b4-7d4f029d9149 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:30.996322 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08490b-dbad-4293-a00a-43dbffc35989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:30.996975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08490b-dbad-4293-a00a-43dbffc35989 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:30.997306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08490b-dbad-4293-a00a-43dbffc35989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:30.997509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08490b-dbad-4293-a00a-43dbffc35989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:30.998178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08490b-dbad-4293-a00a-43dbffc35989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:30.999767 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08490b-dbad-4293-a00a-43dbffc35989 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:31.011018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1279/2557] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:31.017663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c5609-9664-43bd-8db5-4467f9fc8a7a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:31.026004 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c5609-9664-43bd-8db5-4467f9fc8a7a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:31.030693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245400ec-7e82-4fc3-80de-af9e71a18cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:31.031279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245400ec-7e82-4fc3-80de-af9e71a18cac admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.031558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245400ec-7e82-4fc3-80de-af9e71a18cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.031859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245400ec-7e82-4fc3-80de-af9e71a18cac admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.032489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245400ec-7e82-4fc3-80de-af9e71a18cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:31.034099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245400ec-7e82-4fc3-80de-af9e71a18cac admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:31.044560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1279/2558] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:31.048801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08490b-dbad-4293-a00a-43dbffc35989 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:31.059138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08490b-dbad-4293-a00a-43dbffc35989 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:31.067053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90502541-af35-4419-9644-ddde01fe6a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:31.067739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90502541-af35-4419-9644-ddde01fe6a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.068016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90502541-af35-4419-9644-ddde01fe6a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.068346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90502541-af35-4419-9644-ddde01fe6a54 admin admin] PATH_INFO: `/v3/projects/0f358f8d9ea341a3aac6b1f6aea5ba50/users/93491926e31f4c77a461aa6625cae0cf/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.087690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90502541-af35-4419-9644-ddde01fe6a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f358f8d9ea341a3aac6b1f6aea5ba50, user_id=93491926e31f4c77a461aa6625cae0cf, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:31.087809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90502541-af35-4419-9644-ddde01fe6a54 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:31.144212 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1280/2559] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:31 2022] PUT /identity/v3/projects/0f358f8d9ea341a3aac6b1f6aea5ba50/users/93491926e31f4c77a461aa6625cae0cf/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:31.155044 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f22729-dccb-4819-949f-aea3b821567d None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.156634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f22729-dccb-4819-949f-aea3b821567d None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.156634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f22729-dccb-4819-949f-aea3b821567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.191371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-a4f22729-dccb-4819-949f-aea3b821567d None None] MFA Rules not processed for user `93491926e31f4c77a461aa6625cae0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:31.197965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a4f22729-dccb-4819-949f-aea3b821567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:31.257758 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:31.262900 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1280/2560] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:31.271591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90502541-af35-4419-9644-ddde01fe6a54 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:31.286832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:31.333604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90502541-af35-4419-9644-ddde01fe6a54 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:31.338481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af22e3-b4d9-41f2-a48d-2fe8c460902e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:31.339071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af22e3-b4d9-41f2-a48d-2fe8c460902e admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.339340 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af22e3-b4d9-41f2-a48d-2fe8c460902e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.339598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af22e3-b4d9-41f2-a48d-2fe8c460902e admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.340216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af22e3-b4d9-41f2-a48d-2fe8c460902e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:31.342270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af22e3-b4d9-41f2-a48d-2fe8c460902e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:31.458489 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1281/2561] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:31 2022] POST /identity/v3/projects => generated 393 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:31.463140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f22729-dccb-4819-949f-aea3b821567d None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:31.474282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f22729-dccb-4819-949f-aea3b821567d None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:31.537572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753a37a-8905-4c57-bd44-efa21030fe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:31.538276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753a37a-8905-4c57-bd44-efa21030fe4e admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.538464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753a37a-8905-4c57-bd44-efa21030fe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.538738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753a37a-8905-4c57-bd44-efa21030fe4e admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.539361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753a37a-8905-4c57-bd44-efa21030fe4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:31.541313 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753a37a-8905-4c57-bd44-efa21030fe4e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:31.621271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1281/2562] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:31 2022] POST /identity/v3/users => generated 303 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:31.622444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af22e3-b4d9-41f2-a48d-2fe8c460902e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:31.633135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af22e3-b4d9-41f2-a48d-2fe8c460902e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:31.637903 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ab95c-f9ba-4486-9863-59969ffe5924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:31.638524 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab95c-f9ba-4486-9863-59969ffe5924 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.638804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab95c-f9ba-4486-9863-59969ffe5924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.639196 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab95c-f9ba-4486-9863-59969ffe5924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.639953 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ab95c-f9ba-4486-9863-59969ffe5924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:31.641593 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ab95c-f9ba-4486-9863-59969ffe5924 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:31.661075 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1282/2563] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:31.664182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753a37a-8905-4c57-bd44-efa21030fe4e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:31.675606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753a37a-8905-4c57-bd44-efa21030fe4e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:31.680477 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeeb48-470f-44c3-9c89-9b35d0b6cb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:31.680981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeeb48-470f-44c3-9c89-9b35d0b6cb28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.681256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeeb48-470f-44c3-9c89-9b35d0b6cb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.681524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeeb48-470f-44c3-9c89-9b35d0b6cb28 admin admin] PATH_INFO: `/v3/projects/537b7ba518f7470785efc6384c170ad3/users/cfeec484f24e40deb00167eff71d07e2/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.696943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aeeb48-470f-44c3-9c89-9b35d0b6cb28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=537b7ba518f7470785efc6384c170ad3, user_id=cfeec484f24e40deb00167eff71d07e2, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:31.699560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aeeb48-470f-44c3-9c89-9b35d0b6cb28 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:31.766447 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1282/2564] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:31 2022] PUT /identity/v3/projects/537b7ba518f7470785efc6384c170ad3/users/cfeec484f24e40deb00167eff71d07e2/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:31.770360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ab95c-f9ba-4486-9863-59969ffe5924 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:31.780366 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ab95c-f9ba-4486-9863-59969ffe5924 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:31.839269 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae561f-d424-4936-aa88-b9e4292a9b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:31.839860 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae561f-d424-4936-aa88-b9e4292a9b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.840130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae561f-d424-4936-aa88-b9e4292a9b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.840393 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae561f-d424-4936-aa88-b9e4292a9b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.841133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae561f-d424-4936-aa88-b9e4292a9b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:31.842705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae561f-d424-4936-aa88-b9e4292a9b63 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:31.856127 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1283/2565] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:31.861373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeeb48-470f-44c3-9c89-9b35d0b6cb28 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:31.872475 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeeb48-470f-44c3-9c89-9b35d0b6cb28 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:31.877195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad944b6e-769f-45d7-8205-3bdb2fb03d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:31.877831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad944b6e-769f-45d7-8205-3bdb2fb03d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.878105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad944b6e-769f-45d7-8205-3bdb2fb03d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.878375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad944b6e-769f-45d7-8205-3bdb2fb03d94 admin admin] PATH_INFO: `/v3/projects/537b7ba518f7470785efc6384c170ad3/users/cfeec484f24e40deb00167eff71d07e2/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.880249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad944b6e-769f-45d7-8205-3bdb2fb03d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=537b7ba518f7470785efc6384c170ad3, user_id=cfeec484f24e40deb00167eff71d07e2, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:31.882920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad944b6e-769f-45d7-8205-3bdb2fb03d94 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:31.893502 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1283/2566] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:31 2022] PUT /identity/v3/projects/537b7ba518f7470785efc6384c170ad3/users/cfeec484f24e40deb00167eff71d07e2/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:31.903483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92a638-0d44-46ed-92f3-954a957aa026 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:31.903774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92a638-0d44-46ed-92f3-954a957aa026 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:31.904047 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92a638-0d44-46ed-92f3-954a957aa026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:31.931334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1d92a638-0d44-46ed-92f3-954a957aa026 None None] MFA Rules not processed for user `cfeec484f24e40deb00167eff71d07e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:31.932552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1d92a638-0d44-46ed-92f3-954a957aa026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:31.998804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1284/2567] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:31 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:16:32.009471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad944b6e-769f-45d7-8205-3bdb2fb03d94 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:32.011161 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ad944b6e-769f-45d7-8205-3bdb2fb03d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:32.069331 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad944b6e-769f-45d7-8205-3bdb2fb03d94 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:32.073097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09663f32-821c-4eb8-a6a0-4ddfb64f1f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:32.073696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09663f32-821c-4eb8-a6a0-4ddfb64f1f4e service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:32.073968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09663f32-821c-4eb8-a6a0-4ddfb64f1f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:32.074231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09663f32-821c-4eb8-a6a0-4ddfb64f1f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:32.084297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09663f32-821c-4eb8-a6a0-4ddfb64f1f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:32.085741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09663f32-821c-4eb8-a6a0-4ddfb64f1f4e service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:32.095826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1284/2568] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:32 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:32.146590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92a638-0d44-46ed-92f3-954a957aa026 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:32.147986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1d92a638-0d44-46ed-92f3-954a957aa026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:32.160131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92a638-0d44-46ed-92f3-954a957aa026 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:32.163793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17539ff2-a5f5-4ba7-a0e4-69c041b7b38b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:32.164384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17539ff2-a5f5-4ba7-a0e4-69c041b7b38b service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:32.164702 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17539ff2-a5f5-4ba7-a0e4-69c041b7b38b service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:32.165012 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17539ff2-a5f5-4ba7-a0e4-69c041b7b38b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:32.174799 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:32.227893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17539ff2-a5f5-4ba7-a0e4-69c041b7b38b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:32.227893 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17539ff2-a5f5-4ba7-a0e4-69c041b7b38b service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:32.233485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1285/2569] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:36.987813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09663f32-821c-4eb8-a6a0-4ddfb64f1f4e service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:37.039718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:37.049922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09663f32-821c-4eb8-a6a0-4ddfb64f1f4e service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:37.054439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:37.055060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:37.055328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:37.055590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] PATH_INFO: `/v3/users/93491926e31f4c77a461aa6625cae0cf` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:37.056797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93491926e31f4c77a461aa6625cae0cf)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:37.058733 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:37.236359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93491926e31f4c77a461aa6625cae0cf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.308360 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93491926e31f4c77a461aa6625cae0cf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.315480 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93491926e31f4c77a461aa6625cae0cf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.316748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93491926e31f4c77a461aa6625cae0cf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.317357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93491926e31f4c77a461aa6625cae0cf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.335117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1285/2570] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:36 2022] DELETE /identity/v3/users/93491926e31f4c77a461aa6625cae0cf => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:37.339618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee9d7b-078b-41dc-93cd-17a6c1708162 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:37.340085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee9d7b-078b-41dc-93cd-17a6c1708162 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:37.340175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee9d7b-078b-41dc-93cd-17a6c1708162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:37.364042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-bfee9d7b-078b-41dc-93cd-17a6c1708162 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:37.365472 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bfee9d7b-078b-41dc-93cd-17a6c1708162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:37.428907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1286/2571] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:37.445969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:37.447381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:37.509371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e9180-9126-4229-b49c-00ad3f047f92 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:37.513168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aa55c-235e-446d-85a3-91a538ec019d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:37.513772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aa55c-235e-446d-85a3-91a538ec019d service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:37.514049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aa55c-235e-446d-85a3-91a538ec019d service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:37.514315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aa55c-235e-446d-85a3-91a538ec019d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:37.524798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40aa55c-235e-446d-85a3-91a538ec019d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:37.529674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40aa55c-235e-446d-85a3-91a538ec019d service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:37.536233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1286/2572] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:37.627143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee9d7b-078b-41dc-93cd-17a6c1708162 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:37.629313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-bfee9d7b-078b-41dc-93cd-17a6c1708162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:37.641013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee9d7b-078b-41dc-93cd-17a6c1708162 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:37.646190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:37.646798 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:37.647077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:37.647346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] PATH_INFO: `/v3/users/09cef90067284811a59c0691a3a12e58` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:37.648457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=09cef90067284811a59c0691a3a12e58)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:37.650539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:37.839900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09cef90067284811a59c0691a3a12e58'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.898035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09cef90067284811a59c0691a3a12e58'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.905912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aa55c-235e-446d-85a3-91a538ec019d service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:37.906786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a40aa55c-235e-446d-85a3-91a538ec019d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:37.907219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09cef90067284811a59c0691a3a12e58'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.907977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09cef90067284811a59c0691a3a12e58'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.909280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-ee83e47c-97ff-4a0c-a1dc-c8a7824cfd4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09cef90067284811a59c0691a3a12e58'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:37.919943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40aa55c-235e-446d-85a3-91a538ec019d service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:37.926916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:37.927807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:37.928694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:37.928694 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] PATH_INFO: `/v3/projects/0f358f8d9ea341a3aac6b1f6aea5ba50` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:37.929914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f358f8d9ea341a3aac6b1f6aea5ba50)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:37.932158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:37.940914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1287/2573] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:37 2022] DELETE /identity/v3/users/09cef90067284811a59c0691a3a12e58 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:37.945853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba3649-c63b-4460-a28a-2844b941b3ba None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:37.946143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba3649-c63b-4460-a28a-2844b941b3ba None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:37.946411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba3649-c63b-4460-a28a-2844b941b3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:37.965739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d0ba3649-c63b-4460-a28a-2844b941b3ba None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:37.966931 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d0ba3649-c63b-4460-a28a-2844b941b3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:37.985652 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:38.026151 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1288/2574] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:16:38.035855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba3649-c63b-4460-a28a-2844b941b3ba None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:38.036277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f358f8d9ea341a3aac6b1f6aea5ba50', 'request_id': 'req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.037178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d0ba3649-c63b-4460-a28a-2844b941b3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:38.042035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f358f8d9ea341a3aac6b1f6aea5ba50', 'request_id': 'req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.094135 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba3649-c63b-4460-a28a-2844b941b3ba None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:38.098204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f358f8d9ea341a3aac6b1f6aea5ba50', 'request_id': 'req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.107666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:38.113674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] The token cache is being invalidate because project 0f358f8d9ea341a3aac6b1f6aea5ba50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:38.114077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.114850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.116430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1287/2575] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:37 2022] DELETE /identity/v3/projects/0f358f8d9ea341a3aac6b1f6aea5ba50 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:38.120413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:38.121795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:38.153574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:38.154195 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:38.154476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:38.154751 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:38.156590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:38.189381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a78ab-76c0-42c4-8117-9f0fe0c05bb4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:38.194782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:38.195407 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:38.195680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:38.195948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] PATH_INFO: `/v3/users/cfeec484f24e40deb00167eff71d07e2` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:38.197104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfeec484f24e40deb00167eff71d07e2)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:38.199107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:38.218269 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:38.219691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:38.229613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1289/2576] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:16:38.455082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfeec484f24e40deb00167eff71d07e2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.549488 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfeec484f24e40deb00167eff71d07e2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.556984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfeec484f24e40deb00167eff71d07e2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.558098 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfeec484f24e40deb00167eff71d07e2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.558708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfeec484f24e40deb00167eff71d07e2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.576853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1288/2577] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:38 2022] DELETE /identity/v3/users/cfeec484f24e40deb00167eff71d07e2 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:38.681805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:38.683249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:38.707035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30a7b7-2cf5-4411-bc8e-5fa930b17ad6 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:38.739493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:38.767317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:38.767944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:38.768220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:38.768490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] PATH_INFO: `/v3/projects/d9ea80e3da8a4ceb9ef0b7011f7ea374` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:38.769643 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ea80e3da8a4ceb9ef0b7011f7ea374)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:38.771596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:38.866956 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ea80e3da8a4ceb9ef0b7011f7ea374', 'request_id': 'req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.872583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ea80e3da8a4ceb9ef0b7011f7ea374', 'request_id': 'req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.943574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ea80e3da8a4ceb9ef0b7011f7ea374', 'request_id': 'req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.955541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] The token cache is being invalidate because project d9ea80e3da8a4ceb9ef0b7011f7ea374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:38.956080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.956775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:38.958339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1290/2578] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:38 2022] DELETE /identity/v3/projects/d9ea80e3da8a4ceb9ef0b7011f7ea374 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:38.962393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:38.963772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:38.977295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101cb3e-0669-4061-8fc3-268b69e30bcf admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:39.033552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:39.034169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.034450 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.035246 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] PATH_INFO: `/v3/users/4111366e46444f188098079090808841` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.036363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4111366e46444f188098079090808841)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:39.038309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:39.048138 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:39.049632 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:39.063670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663d196-d1f6-4d1f-a09c-af6bc1e9b1eb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:39.068086 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:39.068933 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:39.069582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.069897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.070124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] PATH_INFO: `/v3/projects/537b7ba518f7470785efc6384c170ad3` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.071286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=537b7ba518f7470785efc6384c170ad3)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:39.073229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:39.210615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '537b7ba518f7470785efc6384c170ad3', 'request_id': 'req-448cc3df-80b2-4044-ba55-45f257fffc39'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.213154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4111366e46444f188098079090808841'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.216597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '537b7ba518f7470785efc6384c170ad3', 'request_id': 'req-448cc3df-80b2-4044-ba55-45f257fffc39'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.274870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4111366e46444f188098079090808841'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.282249 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4111366e46444f188098079090808841'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.282966 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4111366e46444f188098079090808841'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.283601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-620c08c5-a05e-4601-85bf-4e33c2e73a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4111366e46444f188098079090808841'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.302333 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1289/2579] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:38 2022] DELETE /identity/v3/users/4111366e46444f188098079090808841 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:39.320742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '537b7ba518f7470785efc6384c170ad3', 'request_id': 'req-448cc3df-80b2-4044-ba55-45f257fffc39'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.333796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] The token cache is being invalidate because project 537b7ba518f7470785efc6384c170ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:39.334126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.334781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-448cc3df-80b2-4044-ba55-45f257fffc39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.336295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1291/2580] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:39 2022] DELETE /identity/v3/projects/537b7ba518f7470785efc6384c170ad3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:39.341445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a8e8-eebb-4d5a-a8ea-b783fd956943 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.341752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a8e8-eebb-4d5a-a8ea-b783fd956943 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.342021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a8e8-eebb-4d5a-a8ea-b783fd956943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.362749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c906a8e8-eebb-4d5a-a8ea-b783fd956943 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:39.363967 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c906a8e8-eebb-4d5a-a8ea-b783fd956943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:39.424158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1290/2581] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:39.433295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a603b14-9e77-4ea1-9ba7-77e5892863c6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.433572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a603b14-9e77-4ea1-9ba7-77e5892863c6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.436714 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a603b14-9e77-4ea1-9ba7-77e5892863c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.455425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5a603b14-9e77-4ea1-9ba7-77e5892863c6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:39.459793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5a603b14-9e77-4ea1-9ba7-77e5892863c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:39.468209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1292/2582] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:39.473545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906a8e8-eebb-4d5a-a8ea-b783fd956943 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:39.484795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906a8e8-eebb-4d5a-a8ea-b783fd956943 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:39.490075 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a85a7-e0ea-4023-9be1-60040b94c138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:39.490803 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a85a7-e0ea-4023-9be1-60040b94c138 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.490939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a85a7-e0ea-4023-9be1-60040b94c138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.491237 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a85a7-e0ea-4023-9be1-60040b94c138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.491888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a85a7-e0ea-4023-9be1-60040b94c138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:16:39.492183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a85a7-e0ea-4023-9be1-60040b94c138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:39.493895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a85a7-e0ea-4023-9be1-60040b94c138 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:39.517750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1291/2583] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:16:39.525328 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a603b14-9e77-4ea1-9ba7-77e5892863c6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:39.535560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a603b14-9e77-4ea1-9ba7-77e5892863c6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:39.541098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9112b19-1a54-4836-86e7-2ffb355b6357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:39.541694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9112b19-1a54-4836-86e7-2ffb355b6357 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.541964 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9112b19-1a54-4836-86e7-2ffb355b6357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.542224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9112b19-1a54-4836-86e7-2ffb355b6357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.542843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9112b19-1a54-4836-86e7-2ffb355b6357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:39.544755 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9112b19-1a54-4836-86e7-2ffb355b6357 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:39.671122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1293/2584] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:39 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:39.675304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a85a7-e0ea-4023-9be1-60040b94c138 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:39.686662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a85a7-e0ea-4023-9be1-60040b94c138 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:39.689528 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9112b19-1a54-4836-86e7-2ffb355b6357 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:39.691875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a9112b19-1a54-4836-86e7-2ffb355b6357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:39.712963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:39.750029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6254a-3f89-4bbd-ae44-51878eb1a428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:39.750666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6254a-3f89-4bbd-ae44-51878eb1a428 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.750938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6254a-3f89-4bbd-ae44-51878eb1a428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.751232 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6254a-3f89-4bbd-ae44-51878eb1a428 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.751858 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6254a-3f89-4bbd-ae44-51878eb1a428 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:39.754361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6254a-3f89-4bbd-ae44-51878eb1a428 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:39.760618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9112b19-1a54-4836-86e7-2ffb355b6357 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:39.767207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:39.767832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.768116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.768382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] PATH_INFO: `/v3/projects/79b9691590c3490f94757b7055714ea1` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.770145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] RBAC: Authorizing `identity:delete_project(project_id=79b9691590c3490f94757b7055714ea1)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:39.772041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:39.818738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1292/2585] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:39 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:39.822512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6254a-3f89-4bbd-ae44-51878eb1a428 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:39.833034 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6254a-3f89-4bbd-ae44-51878eb1a428 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:39.837557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885311e3-ed66-40a2-9754-b81ba06bbdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:39.838126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885311e3-ed66-40a2-9754-b81ba06bbdc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.838402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885311e3-ed66-40a2-9754-b81ba06bbdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.838669 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885311e3-ed66-40a2-9754-b81ba06bbdc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.839324 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885311e3-ed66-40a2-9754-b81ba06bbdc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:39.840986 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885311e3-ed66-40a2-9754-b81ba06bbdc4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:39.851522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1293/2586] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:39.855292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885311e3-ed66-40a2-9754-b81ba06bbdc4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:39.860099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:39.865254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885311e3-ed66-40a2-9754-b81ba06bbdc4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:39.869915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3a041-c0ac-4c85-918b-c968e1e20ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:39.870487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3a041-c0ac-4c85-918b-c968e1e20ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.870765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3a041-c0ac-4c85-918b-c968e1e20ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.871029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3a041-c0ac-4c85-918b-c968e1e20ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.871656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3a041-c0ac-4c85-918b-c968e1e20ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:39.873317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3a041-c0ac-4c85-918b-c968e1e20ba4 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:39.883883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1294/2587] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:39.887857 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3a041-c0ac-4c85-918b-c968e1e20ba4 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:39.887990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79b9691590c3490f94757b7055714ea1', 'request_id': 'req-5a0b20f5-c385-43ed-a461-cf2375c3997e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.893527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79b9691590c3490f94757b7055714ea1', 'request_id': 'req-5a0b20f5-c385-43ed-a461-cf2375c3997e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.897995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3a041-c0ac-4c85-918b-c968e1e20ba4 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:39.951715 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fc276-bf4f-483d-a705-04f18bc69dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:39.952353 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fc276-bf4f-483d-a705-04f18bc69dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.952672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fc276-bf4f-483d-a705-04f18bc69dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.952939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fc276-bf4f-483d-a705-04f18bc69dfb admin admin] PATH_INFO: `/v3/projects/d3c8986c30914529ac94becd8e0ebfe3/users/241a3972e0a14223ac8a4ee38996a492/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:39.967979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fc276-bf4f-483d-a705-04f18bc69dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c8986c30914529ac94becd8e0ebfe3, user_id=241a3972e0a14223ac8a4ee38996a492, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:39.970571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fc276-bf4f-483d-a705-04f18bc69dfb admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:39.977978 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79b9691590c3490f94757b7055714ea1', 'request_id': 'req-5a0b20f5-c385-43ed-a461-cf2375c3997e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.989275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] The token cache is being invalidate because project 79b9691590c3490f94757b7055714ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:39.989564 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.990202 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5a0b20f5-c385-43ed-a461-cf2375c3997e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:39.991687 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1294/2588] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:39 2022] DELETE /identity/v3/projects/79b9691590c3490f94757b7055714ea1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:39.996419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7714d8-b627-450d-a693-87362d196b8c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:39.996819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7714d8-b627-450d-a693-87362d196b8c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:39.997096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7714d8-b627-450d-a693-87362d196b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.017816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1b7714d8-b627-450d-a693-87362d196b8c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:40.019640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1b7714d8-b627-450d-a693-87362d196b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:40.027554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1295/2589] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:16:40.035320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be83c91-5095-4829-b94d-58029933324a None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.039505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be83c91-5095-4829-b94d-58029933324a None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.039623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1295/2590] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:39 2022] PUT /identity/v3/projects/d3c8986c30914529ac94becd8e0ebfe3/users/241a3972e0a14223ac8a4ee38996a492/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:40.040071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be83c91-5095-4829-b94d-58029933324a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.048485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978196a-8a66-4a91-a06b-ac457390bbd7 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.048796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978196a-8a66-4a91-a06b-ac457390bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.049068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978196a-8a66-4a91-a06b-ac457390bbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.057772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6be83c91-5095-4829-b94d-58029933324a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:40.059872 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6be83c91-5095-4829-b94d-58029933324a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:40.076442 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7978196a-8a66-4a91-a06b-ac457390bbd7 None None] MFA Rules not processed for user `241a3972e0a14223ac8a4ee38996a492`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:40.077668 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7978196a-8a66-4a91-a06b-ac457390bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:40.120305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1296/2591] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:16:40.124609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be83c91-5095-4829-b94d-58029933324a None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.134285 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be83c91-5095-4829-b94d-58029933324a None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.139180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc8ba7-002b-4f0e-a757-92b8bc214715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.146591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc8ba7-002b-4f0e-a757-92b8bc214715 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.146591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc8ba7-002b-4f0e-a757-92b8bc214715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.146591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc8ba7-002b-4f0e-a757-92b8bc214715 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.146591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc8ba7-002b-4f0e-a757-92b8bc214715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:16:40.146591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc8ba7-002b-4f0e-a757-92b8bc214715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.146591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc8ba7-002b-4f0e-a757-92b8bc214715 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.149842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1296/2592] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:16:40.154351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978196a-8a66-4a91-a06b-ac457390bbd7 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.155019 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1297/2593] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:16:40.159043 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7978196a-8a66-4a91-a06b-ac457390bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:40.159190 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc8ba7-002b-4f0e-a757-92b8bc214715 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.170377 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc8ba7-002b-4f0e-a757-92b8bc214715 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.172704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978196a-8a66-4a91-a06b-ac457390bbd7 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.178053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab9814-d704-4524-be4a-1995d187e529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.178232 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562dde-2e57-4097-9739-af5532b1d830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.178651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab9814-d704-4524-be4a-1995d187e529 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.178930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab9814-d704-4524-be4a-1995d187e529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.179192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562dde-2e57-4097-9739-af5532b1d830 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.179308 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab9814-d704-4524-be4a-1995d187e529 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.179808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab9814-d704-4524-be4a-1995d187e529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.179922 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562dde-2e57-4097-9739-af5532b1d830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.180209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562dde-2e57-4097-9739-af5532b1d830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.181044 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562dde-2e57-4097-9739-af5532b1d830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.181901 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab9814-d704-4524-be4a-1995d187e529 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.185565 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562dde-2e57-4097-9739-af5532b1d830 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.322856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1298/2594] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:40 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:40.323751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1297/2595] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:40 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:40.327022 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562dde-2e57-4097-9739-af5532b1d830 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.327406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab9814-d704-4524-be4a-1995d187e529 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.392721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab9814-d704-4524-be4a-1995d187e529 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.397775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562dde-2e57-4097-9739-af5532b1d830 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.397912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b5414-e032-441f-b458-5c6b0e643d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.398620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b5414-e032-441f-b458-5c6b0e643d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.398997 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b5414-e032-441f-b458-5c6b0e643d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.399554 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b5414-e032-441f-b458-5c6b0e643d5a admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.400164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b5414-e032-441f-b458-5c6b0e643d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.401955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:40.402642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b5414-e032-441f-b458-5c6b0e643d5a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.404527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc038e53-0e52-43b3-83b5-2f4069ce70c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.405474 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc038e53-0e52-43b3-83b5-2f4069ce70c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.405753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc038e53-0e52-43b3-83b5-2f4069ce70c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.406061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc038e53-0e52-43b3-83b5-2f4069ce70c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.406680 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc038e53-0e52-43b3-83b5-2f4069ce70c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.408832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc038e53-0e52-43b3-83b5-2f4069ce70c7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.505163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1298/2596] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:40 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:40.508702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b5414-e032-441f-b458-5c6b0e643d5a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.518001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b5414-e032-441f-b458-5c6b0e643d5a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.523268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea5091-46b4-44cd-9858-0b1db8a4472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.523833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea5091-46b4-44cd-9858-0b1db8a4472d admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.524110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea5091-46b4-44cd-9858-0b1db8a4472d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.524380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea5091-46b4-44cd-9858-0b1db8a4472d admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.525046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea5091-46b4-44cd-9858-0b1db8a4472d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.526642 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea5091-46b4-44cd-9858-0b1db8a4472d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.534013 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1299/2597] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:40 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:40.539700 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1299/2598] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:40.542179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc038e53-0e52-43b3-83b5-2f4069ce70c7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.543533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea5091-46b4-44cd-9858-0b1db8a4472d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.553859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc038e53-0e52-43b3-83b5-2f4069ce70c7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.554029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea5091-46b4-44cd-9858-0b1db8a4472d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.559207 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c72ce-a494-4068-9182-6a6b02e1b979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.559844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c72ce-a494-4068-9182-6a6b02e1b979 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.560122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c72ce-a494-4068-9182-6a6b02e1b979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.560248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78753b30-aef7-474b-a496-3ea746f34341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.560388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c72ce-a494-4068-9182-6a6b02e1b979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.560853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78753b30-aef7-474b-a496-3ea746f34341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.561051 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c72ce-a494-4068-9182-6a6b02e1b979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.561166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78753b30-aef7-474b-a496-3ea746f34341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.561381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78753b30-aef7-474b-a496-3ea746f34341 admin admin] PATH_INFO: `/v3/projects/7fe4857ceae64872ac59d8d5c8fd4f74/users/c4971fa110c141098b912eb8b980dcdb/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.562619 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c72ce-a494-4068-9182-6a6b02e1b979 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.573326 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1300/2599] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:40.577578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c72ce-a494-4068-9182-6a6b02e1b979 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.579177 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78753b30-aef7-474b-a496-3ea746f34341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fe4857ceae64872ac59d8d5c8fd4f74, user_id=c4971fa110c141098b912eb8b980dcdb, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.582953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78753b30-aef7-474b-a496-3ea746f34341 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.588008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c72ce-a494-4068-9182-6a6b02e1b979 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.592646 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ca11b-4617-40ba-b8c3-5ecf99fcc02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.593226 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ca11b-4617-40ba-b8c3-5ecf99fcc02a admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.593498 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ca11b-4617-40ba-b8c3-5ecf99fcc02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.593765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ca11b-4617-40ba-b8c3-5ecf99fcc02a admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.594394 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ca11b-4617-40ba-b8c3-5ecf99fcc02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.595944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ca11b-4617-40ba-b8c3-5ecf99fcc02a admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.606347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1301/2600] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:40.610281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ca11b-4617-40ba-b8c3-5ecf99fcc02a admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.619485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ca11b-4617-40ba-b8c3-5ecf99fcc02a admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.624029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b6350-b08f-47e9-bf16-8014a48ad0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.624661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b6350-b08f-47e9-bf16-8014a48ad0e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.624938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b6350-b08f-47e9-bf16-8014a48ad0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.625225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b6350-b08f-47e9-bf16-8014a48ad0e3 admin admin] PATH_INFO: `/v3/projects/e015ca1bee2a41d584a337c326ecb278/users/4990990b4ec54df69dfd207dc7d1b947/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.639769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b6350-b08f-47e9-bf16-8014a48ad0e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e015ca1bee2a41d584a337c326ecb278, user_id=4990990b4ec54df69dfd207dc7d1b947, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.642667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b6350-b08f-47e9-bf16-8014a48ad0e3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.652656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1300/2601] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:40 2022] PUT /identity/v3/projects/7fe4857ceae64872ac59d8d5c8fd4f74/users/c4971fa110c141098b912eb8b980dcdb/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:40.656842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78753b30-aef7-474b-a496-3ea746f34341 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.710773 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1302/2602] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:40 2022] PUT /identity/v3/projects/e015ca1bee2a41d584a337c326ecb278/users/4990990b4ec54df69dfd207dc7d1b947/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:40.719413 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78753b30-aef7-474b-a496-3ea746f34341 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.721357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51aa2f4-0e71-442e-ae28-a9f664dccc94 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.721650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51aa2f4-0e71-442e-ae28-a9f664dccc94 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.722124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51aa2f4-0e71-442e-ae28-a9f664dccc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.749305 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c51aa2f4-0e71-442e-ae28-a9f664dccc94 None None] MFA Rules not processed for user `4990990b4ec54df69dfd207dc7d1b947`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:40.750589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c51aa2f4-0e71-442e-ae28-a9f664dccc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:40.768305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:40.776987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a897e4-88c1-434d-a7ae-9bc62d549bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.777593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a897e4-88c1-434d-a7ae-9bc62d549bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.777876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a897e4-88c1-434d-a7ae-9bc62d549bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.778148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a897e4-88c1-434d-a7ae-9bc62d549bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.778788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a897e4-88c1-434d-a7ae-9bc62d549bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.780367 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a897e4-88c1-434d-a7ae-9bc62d549bc8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.790319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1301/2603] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:40.794592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a897e4-88c1-434d-a7ae-9bc62d549bc8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.807218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a897e4-88c1-434d-a7ae-9bc62d549bc8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.809433 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4e78b-0cca-4801-8a47-424b0c824055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.810106 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4e78b-0cca-4801-8a47-424b0c824055 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.810383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4e78b-0cca-4801-8a47-424b0c824055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.810652 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4e78b-0cca-4801-8a47-424b0c824055 admin admin] PATH_INFO: `/v3/projects/7fe4857ceae64872ac59d8d5c8fd4f74/users/c4971fa110c141098b912eb8b980dcdb/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.812664 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4e78b-0cca-4801-8a47-424b0c824055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fe4857ceae64872ac59d8d5c8fd4f74, user_id=c4971fa110c141098b912eb8b980dcdb, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.815476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4e78b-0cca-4801-8a47-424b0c824055 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.817360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1303/2604] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:40.821827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51aa2f4-0e71-442e-ae28-a9f664dccc94 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.827011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1302/2605] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:40 2022] PUT /identity/v3/projects/7fe4857ceae64872ac59d8d5c8fd4f74/users/c4971fa110c141098b912eb8b980dcdb/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:16:40.833588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51aa2f4-0e71-442e-ae28-a9f664dccc94 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:40.840686 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc7432-2480-428c-9c6f-3205fb532231 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.840968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc7432-2480-428c-9c6f-3205fb532231 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.841240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc7432-2480-428c-9c6f-3205fb532231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.871657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d8cc7432-2480-428c-9c6f-3205fb532231 None None] MFA Rules not processed for user `c4971fa110c141098b912eb8b980dcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:40.872917 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d8cc7432-2480-428c-9c6f-3205fb532231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:40.890871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e9cea-cdf8-40e9-8a24-e67b8f20fbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:40.892663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e9cea-cdf8-40e9-8a24-e67b8f20fbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:40.892663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e9cea-cdf8-40e9-8a24-e67b8f20fbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:40.892663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e9cea-cdf8-40e9-8a24-e67b8f20fbf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:40.892916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e9cea-cdf8-40e9-8a24-e67b8f20fbf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:40.894788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e9cea-cdf8-40e9-8a24-e67b8f20fbf8 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:40.939693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1303/2606] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:16:40.950859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc7432-2480-428c-9c6f-3205fb532231 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:40.952187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d8cc7432-2480-428c-9c6f-3205fb532231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:40.961079 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:41.002402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1304/2607] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:40 2022] POST /identity/v3/projects => generated 363 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:41.007061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e9cea-cdf8-40e9-8a24-e67b8f20fbf8 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:41.007529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc7432-2480-428c-9c6f-3205fb532231 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:41.018605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:41.056113 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5e355-493d-4b9e-8a3c-bb86ac3f4d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:41.056801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5e355-493d-4b9e-8a3c-bb86ac3f4d17 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:41.057015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5e355-493d-4b9e-8a3c-bb86ac3f4d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:41.057280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5e355-493d-4b9e-8a3c-bb86ac3f4d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:41.068014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e9cea-cdf8-40e9-8a24-e67b8f20fbf8 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:41.072660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de808d-ddd5-4143-8be9-da8425ded7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:41.073258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de808d-ddd5-4143-8be9-da8425ded7bf admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:41.073548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de808d-ddd5-4143-8be9-da8425ded7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:41.074210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de808d-ddd5-4143-8be9-da8425ded7bf admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:41.075165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de808d-ddd5-4143-8be9-da8425ded7bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:41.077769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de808d-ddd5-4143-8be9-da8425ded7bf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:41.114505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5e355-493d-4b9e-8a3c-bb86ac3f4d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:41.115945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5e355-493d-4b9e-8a3c-bb86ac3f4d17 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:41.124682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1304/2608] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:16:41.187487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1305/2609] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:41 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:16:41.188904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5e355-493d-4b9e-8a3c-bb86ac3f4d17 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:41.200396 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5e355-493d-4b9e-8a3c-bb86ac3f4d17 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:41.205222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40132a51-41d9-48cc-848f-fde76cd1a161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:41.206078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40132a51-41d9-48cc-848f-fde76cd1a161 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:41.211870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40132a51-41d9-48cc-848f-fde76cd1a161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:41.211870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40132a51-41d9-48cc-848f-fde76cd1a161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:41.211870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40132a51-41d9-48cc-848f-fde76cd1a161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:41.211870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40132a51-41d9-48cc-848f-fde76cd1a161 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:41.218957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1305/2610] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:41.223940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de808d-ddd5-4143-8be9-da8425ded7bf admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:41.235092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de808d-ddd5-4143-8be9-da8425ded7bf admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:41.241194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f266e-ef85-403f-b3de-fa06d74442e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:41.241841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f266e-ef85-403f-b3de-fa06d74442e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:41.242146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f266e-ef85-403f-b3de-fa06d74442e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:41.242416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f266e-ef85-403f-b3de-fa06d74442e5 admin admin] PATH_INFO: `/v3/projects/0ab92ab1aedc47ceb179121770f908c5/users/b8dcd9c25b3e42c0b3f0bc07544e7ae2/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:41.257825 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f266e-ef85-403f-b3de-fa06d74442e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab92ab1aedc47ceb179121770f908c5, user_id=b8dcd9c25b3e42c0b3f0bc07544e7ae2, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:41.260483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f266e-ef85-403f-b3de-fa06d74442e5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:41.315955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1306/2611] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:41 2022] PUT /identity/v3/projects/0ab92ab1aedc47ceb179121770f908c5/users/b8dcd9c25b3e42c0b3f0bc07544e7ae2/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:41.319943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40132a51-41d9-48cc-848f-fde76cd1a161 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:41.329968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40132a51-41d9-48cc-848f-fde76cd1a161 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:41.337973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:41.389085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228c706-b370-454e-be9d-8ed8954204e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:41.389728 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228c706-b370-454e-be9d-8ed8954204e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:41.390009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228c706-b370-454e-be9d-8ed8954204e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:41.390352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228c706-b370-454e-be9d-8ed8954204e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:41.390991 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228c706-b370-454e-be9d-8ed8954204e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:41.392806 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228c706-b370-454e-be9d-8ed8954204e7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:41.402593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1306/2612] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:41.406928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f266e-ef85-403f-b3de-fa06d74442e5 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:41.417942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f266e-ef85-403f-b3de-fa06d74442e5 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:41.422578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbf982-af99-4506-a871-f07cac3d46b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:41.423237 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbf982-af99-4506-a871-f07cac3d46b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:41.423515 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbf982-af99-4506-a871-f07cac3d46b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:41.423786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbf982-af99-4506-a871-f07cac3d46b7 admin admin] PATH_INFO: `/v3/projects/0ab92ab1aedc47ceb179121770f908c5/users/b8dcd9c25b3e42c0b3f0bc07544e7ae2/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:41.426312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbf982-af99-4506-a871-f07cac3d46b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab92ab1aedc47ceb179121770f908c5, user_id=b8dcd9c25b3e42c0b3f0bc07544e7ae2, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:41.428970 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbf982-af99-4506-a871-f07cac3d46b7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:41.438990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1307/2613] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:41 2022] PUT /identity/v3/projects/0ab92ab1aedc47ceb179121770f908c5/users/b8dcd9c25b3e42c0b3f0bc07544e7ae2/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:41.450293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54992e2c-ea24-482c-8a1a-9ecce3612a49 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:41.450583 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54992e2c-ea24-482c-8a1a-9ecce3612a49 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:41.450856 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54992e2c-ea24-482c-8a1a-9ecce3612a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:41.480101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-54992e2c-ea24-482c-8a1a-9ecce3612a49 None None] MFA Rules not processed for user `b8dcd9c25b3e42c0b3f0bc07544e7ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:41.481406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-54992e2c-ea24-482c-8a1a-9ecce3612a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:41.557192 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1307/2614] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:41.614664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbf982-af99-4506-a871-f07cac3d46b7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:41.670288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbf982-af99-4506-a871-f07cac3d46b7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:41.674154 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7bd5a-563f-4013-836d-f6b04263e6e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:41.678419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7bd5a-563f-4013-836d-f6b04263e6e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:41.678419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7bd5a-563f-4013-836d-f6b04263e6e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:41.678419 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7bd5a-563f-4013-836d-f6b04263e6e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:41.684640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7bd5a-563f-4013-836d-f6b04263e6e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:41.686070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7bd5a-563f-4013-836d-f6b04263e6e2 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:41.696053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1308/2615] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:41.927845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5872b1-f582-42d5-990e-f981d4bcd33f None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:41.928117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5872b1-f582-42d5-990e-f981d4bcd33f None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:41.928388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5872b1-f582-42d5-990e-f981d4bcd33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:41.947655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-ef5872b1-f582-42d5-990e-f981d4bcd33f None None] MFA Rules not processed for user `0c911d938fd64f25844903809d56b263`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:41.948940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ef5872b1-f582-42d5-990e-f981d4bcd33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:42.006348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1308/2616] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:42.341998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7bd5a-563f-4013-836d-f6b04263e6e2 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:42.343527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ecd7bd5a-563f-4013-836d-f6b04263e6e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:42.364257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:42.402312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7bd5a-563f-4013-836d-f6b04263e6e2 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:42.406126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4330f-7e11-4454-be27-b38e58cfc6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:42.406805 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4330f-7e11-4454-be27-b38e58cfc6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:42.414610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4330f-7e11-4454-be27-b38e58cfc6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:42.414610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4330f-7e11-4454-be27-b38e58cfc6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:42.414610 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d3e4330f-7e11-4454-be27-b38e58cfc6d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:42.419169 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4330f-7e11-4454-be27-b38e58cfc6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:42.420598 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4330f-7e11-4454-be27-b38e58cfc6d9 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:42.428370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1309/2617] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:42.790162 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5872b1-f582-42d5-990e-f981d4bcd33f None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:42.963891 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:42.986818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5872b1-f582-42d5-990e-f981d4bcd33f None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:42.991973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:42.992731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:42.993030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:42.993304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] PATH_INFO: `/v3/users/4990990b4ec54df69dfd207dc7d1b947` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:42.994424 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4990990b4ec54df69dfd207dc7d1b947)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:42.996402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:43.257914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4990990b4ec54df69dfd207dc7d1b947'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:43.321176 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4990990b4ec54df69dfd207dc7d1b947'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:43.330146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4990990b4ec54df69dfd207dc7d1b947'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:43.331624 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4990990b4ec54df69dfd207dc7d1b947'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:43.332240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4990990b4ec54df69dfd207dc7d1b947'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:43.352222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1309/2618] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:42 2022] DELETE /identity/v3/users/4990990b4ec54df69dfd207dc7d1b947 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:43.356993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83d445-d265-4af3-b810-47e798aff72e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:43.361145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83d445-d265-4af3-b810-47e798aff72e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:43.361145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83d445-d265-4af3-b810-47e798aff72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:43.376774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5d83d445-d265-4af3-b810-47e798aff72e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:43.377974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5d83d445-d265-4af3-b810-47e798aff72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:43.435456 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1310/2619] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:43.446211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:43.447590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:43.507164 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db9526-7e85-4de8-93bd-b64cfd58e31a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:43.510943 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156f95c-6e55-43f4-a3fc-cbccc751d4e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:43.511546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156f95c-6e55-43f4-a3fc-cbccc751d4e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:43.511847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156f95c-6e55-43f4-a3fc-cbccc751d4e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:43.512076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156f95c-6e55-43f4-a3fc-cbccc751d4e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:43.524807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156f95c-6e55-43f4-a3fc-cbccc751d4e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:43.524807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156f95c-6e55-43f4-a3fc-cbccc751d4e8 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:43.534746 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1310/2620] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:44.003868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83d445-d265-4af3-b810-47e798aff72e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:44.006290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5d83d445-d265-4af3-b810-47e798aff72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:44.029021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83d445-d265-4af3-b810-47e798aff72e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:44.034449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:44.035644 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:44.035917 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:44.036219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] PATH_INFO: `/v3/projects/e015ca1bee2a41d584a337c326ecb278` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:44.037478 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e015ca1bee2a41d584a337c326ecb278)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:44.044783 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:44.127710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e015ca1bee2a41d584a337c326ecb278', 'request_id': 'req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.133379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e015ca1bee2a41d584a337c326ecb278', 'request_id': 'req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.207262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e015ca1bee2a41d584a337c326ecb278', 'request_id': 'req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.218954 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] The token cache is being invalidate because project e015ca1bee2a41d584a337c326ecb278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:44.219243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.219887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.221696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1311/2621] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:44 2022] DELETE /identity/v3/projects/e015ca1bee2a41d584a337c326ecb278 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:44.226430 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156f95c-6e55-43f4-a3fc-cbccc751d4e8 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:44.236138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156f95c-6e55-43f4-a3fc-cbccc751d4e8 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:44.294336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:44.294964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:44.295240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:44.295508 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] PATH_INFO: `/v3/users/b8dcd9c25b3e42c0b3f0bc07544e7ae2` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:44.296697 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8dcd9c25b3e42c0b3f0bc07544e7ae2)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:44.298606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:44.300259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:44.460241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8dcd9c25b3e42c0b3f0bc07544e7ae2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.521155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8dcd9c25b3e42c0b3f0bc07544e7ae2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.528615 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8dcd9c25b3e42c0b3f0bc07544e7ae2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.529193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8dcd9c25b3e42c0b3f0bc07544e7ae2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.529820 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-27b4118b-00ed-495b-86a1-185b6fcb483e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8dcd9c25b3e42c0b3f0bc07544e7ae2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:44.546621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1311/2622] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:44 2022] DELETE /identity/v3/users/b8dcd9c25b3e42c0b3f0bc07544e7ae2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:44.974198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:44.975655 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:44.989835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057962e-a6fb-4eae-a067-ea3b0f1a4acb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:45.032271 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:45.047176 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:45.047797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.048073 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.048338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] PATH_INFO: `/v3/projects/0ab92ab1aedc47ceb179121770f908c5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.049469 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ab92ab1aedc47ceb179121770f908c5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:45.052040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:45.184002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ab92ab1aedc47ceb179121770f908c5', 'request_id': 'req-1cd4b247-0d53-46b6-b45b-54011b54e9da'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:45.189715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ab92ab1aedc47ceb179121770f908c5', 'request_id': 'req-1cd4b247-0d53-46b6-b45b-54011b54e9da'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:45.263509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ab92ab1aedc47ceb179121770f908c5', 'request_id': 'req-1cd4b247-0d53-46b6-b45b-54011b54e9da'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:45.280288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] The token cache is being invalidate because project 0ab92ab1aedc47ceb179121770f908c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:16:45.280737 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:45.281406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1cd4b247-0d53-46b6-b45b-54011b54e9da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:16:45.282994 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1312/2623] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:16:44 2022] DELETE /identity/v3/projects/0ab92ab1aedc47ceb179121770f908c5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:45.288435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2039e1-3419-478f-a9bb-8c7716dfe60e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.288936 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2039e1-3419-478f-a9bb-8c7716dfe60e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.289874 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2039e1-3419-478f-a9bb-8c7716dfe60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.308663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-4c2039e1-3419-478f-a9bb-8c7716dfe60e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:45.309827 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-4c2039e1-3419-478f-a9bb-8c7716dfe60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:45.366507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1312/2624] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:45.374122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b165de9-771f-442a-a7b2-0c50323cad02 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.374407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b165de9-771f-442a-a7b2-0c50323cad02 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.374710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b165de9-771f-442a-a7b2-0c50323cad02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.396428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-9b165de9-771f-442a-a7b2-0c50323cad02 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:45.397664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9b165de9-771f-442a-a7b2-0c50323cad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:45.406028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1313/2625] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:45.413240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2039e1-3419-478f-a9bb-8c7716dfe60e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:45.422902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2039e1-3419-478f-a9bb-8c7716dfe60e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:45.427524 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39ef9d-408b-48c2-a9f7-6a82074c7818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:45.428110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39ef9d-408b-48c2-a9f7-6a82074c7818 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.428384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39ef9d-408b-48c2-a9f7-6a82074c7818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.428685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39ef9d-408b-48c2-a9f7-6a82074c7818 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.429327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39ef9d-408b-48c2-a9f7-6a82074c7818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:16:45.429736 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39ef9d-408b-48c2-a9f7-6a82074c7818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:45.431322 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39ef9d-408b-48c2-a9f7-6a82074c7818 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:45.443130 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1313/2626] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:16:45.447476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b165de9-771f-442a-a7b2-0c50323cad02 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:45.457599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b165de9-771f-442a-a7b2-0c50323cad02 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:45.462444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b742da6-8174-4c6f-8d11-54d4c0c24a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:45.463184 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742da6-8174-4c6f-8d11-54d4c0c24a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.463288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742da6-8174-4c6f-8d11-54d4c0c24a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.463551 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742da6-8174-4c6f-8d11-54d4c0c24a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.464182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b742da6-8174-4c6f-8d11-54d4c0c24a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:45.466102 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b742da6-8174-4c6f-8d11-54d4c0c24a09 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:45.561844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1314/2627] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:45 2022] POST /identity/v3/projects => generated 409 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:45.566257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39ef9d-408b-48c2-a9f7-6a82074c7818 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:45.577713 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39ef9d-408b-48c2-a9f7-6a82074c7818 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:45.633552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028458fb-a1c6-4caa-8622-a91eb9f0579d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:45.634151 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028458fb-a1c6-4caa-8622-a91eb9f0579d admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.634428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028458fb-a1c6-4caa-8622-a91eb9f0579d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.634696 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028458fb-a1c6-4caa-8622-a91eb9f0579d admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.635314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028458fb-a1c6-4caa-8622-a91eb9f0579d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:45.637251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028458fb-a1c6-4caa-8622-a91eb9f0579d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:45.648569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:45.722681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1314/2628] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:45 2022] POST /identity/v3/users => generated 305 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:45.722877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b742da6-8174-4c6f-8d11-54d4c0c24a09 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:45.725252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b742da6-8174-4c6f-8d11-54d4c0c24a09 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:45.729890 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fcb96-5166-48bc-bb0c-5d7529ce7eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:45.730477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fcb96-5166-48bc-bb0c-5d7529ce7eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.730923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fcb96-5166-48bc-bb0c-5d7529ce7eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.731189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fcb96-5166-48bc-bb0c-5d7529ce7eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.731825 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fcb96-5166-48bc-bb0c-5d7529ce7eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:45.733438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fcb96-5166-48bc-bb0c-5d7529ce7eeb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:45.743110 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1315/2629] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:45.746979 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028458fb-a1c6-4caa-8622-a91eb9f0579d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:45.759572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028458fb-a1c6-4caa-8622-a91eb9f0579d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:45.764188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b58a47-43a9-40f3-a9cc-c747d3e4b2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:45.764789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b58a47-43a9-40f3-a9cc-c747d3e4b2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.765070 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b58a47-43a9-40f3-a9cc-c747d3e4b2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.765470 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b58a47-43a9-40f3-a9cc-c747d3e4b2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.766123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b58a47-43a9-40f3-a9cc-c747d3e4b2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:45.767681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b58a47-43a9-40f3-a9cc-c747d3e4b2ae admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:45.778251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1315/2630] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:45.782306 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fcb96-5166-48bc-bb0c-5d7529ce7eeb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:45.792290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fcb96-5166-48bc-bb0c-5d7529ce7eeb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:45.797150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafadf13-9e53-4b40-966a-cffbe79b5383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:45.797906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafadf13-9e53-4b40-966a-cffbe79b5383 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.798180 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafadf13-9e53-4b40-966a-cffbe79b5383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.798448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafadf13-9e53-4b40-966a-cffbe79b5383 admin admin] PATH_INFO: `/v3/projects/7541e89df6f84901a968036313b383be/users/7541c41d8d6f473e9c8af7126570569b/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.814035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafadf13-9e53-4b40-966a-cffbe79b5383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7541e89df6f84901a968036313b383be, user_id=7541c41d8d6f473e9c8af7126570569b, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:45.817355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafadf13-9e53-4b40-966a-cffbe79b5383 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:45.891545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1316/2631] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:45 2022] PUT /identity/v3/projects/7541e89df6f84901a968036313b383be/users/7541c41d8d6f473e9c8af7126570569b/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:45.903518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89ea9-22e4-45ad-aaf0-75fa9f056048 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:45.903798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89ea9-22e4-45ad-aaf0-75fa9f056048 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:45.904104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89ea9-22e4-45ad-aaf0-75fa9f056048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:45.931645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-14b89ea9-22e4-45ad-aaf0-75fa9f056048 None None] MFA Rules not processed for user `7541c41d8d6f473e9c8af7126570569b`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:45.932887 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-14b89ea9-22e4-45ad-aaf0-75fa9f056048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:45.995144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1316/2632] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:45 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:45.999776 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafadf13-9e53-4b40-966a-cffbe79b5383 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:46.063354 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafadf13-9e53-4b40-966a-cffbe79b5383 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:46.068832 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a9c23-f277-427d-96d2-ae637aa95dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:46.069454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a9c23-f277-427d-96d2-ae637aa95dde admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:46.069716 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a9c23-f277-427d-96d2-ae637aa95dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:46.069977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a9c23-f277-427d-96d2-ae637aa95dde admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:46.070623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a9c23-f277-427d-96d2-ae637aa95dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:46.072575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a9c23-f277-427d-96d2-ae637aa95dde admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:46.088227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:46.162317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1317/2633] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:16:45 2022] POST /identity/v3/projects => generated 407 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:46.166636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89ea9-22e4-45ad-aaf0-75fa9f056048 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:46.178010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89ea9-22e4-45ad-aaf0-75fa9f056048 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:46.236484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6bf2f-9f3a-45c9-a54b-80e4f941ff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:46.237134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6bf2f-9f3a-45c9-a54b-80e4f941ff15 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:46.237404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6bf2f-9f3a-45c9-a54b-80e4f941ff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:46.238199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6bf2f-9f3a-45c9-a54b-80e4f941ff15 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:46.238977 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6bf2f-9f3a-45c9-a54b-80e4f941ff15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:46.241021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6bf2f-9f3a-45c9-a54b-80e4f941ff15 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:46.367834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1317/2634] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:16:46 2022] POST /identity/v3/users => generated 310 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:16:46.371976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a9c23-f277-427d-96d2-ae637aa95dde admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:46.382705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a9c23-f277-427d-96d2-ae637aa95dde admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:46.387258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90503fdf-de47-40ca-a133-c34786d20b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:46.387835 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90503fdf-de47-40ca-a133-c34786d20b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:46.388160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90503fdf-de47-40ca-a133-c34786d20b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:46.388408 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90503fdf-de47-40ca-a133-c34786d20b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:46.389059 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90503fdf-de47-40ca-a133-c34786d20b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:46.390680 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90503fdf-de47-40ca-a133-c34786d20b43 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:46.402512 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1318/2635] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:16:46.406662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6bf2f-9f3a-45c9-a54b-80e4f941ff15 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:46.416814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6bf2f-9f3a-45c9-a54b-80e4f941ff15 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:46.421683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37543f21-0188-4b3e-9f9f-a0763f57caa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:46.422355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543f21-0188-4b3e-9f9f-a0763f57caa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:46.422629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543f21-0188-4b3e-9f9f-a0763f57caa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:46.422915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543f21-0188-4b3e-9f9f-a0763f57caa7 admin admin] PATH_INFO: `/v3/projects/8438f3dd16ff433492da5cf51302d0e5/users/5abed076a48b4e239cb9e2021f0960f9/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:46.438147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37543f21-0188-4b3e-9f9f-a0763f57caa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8438f3dd16ff433492da5cf51302d0e5, user_id=5abed076a48b4e239cb9e2021f0960f9, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:46.441428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37543f21-0188-4b3e-9f9f-a0763f57caa7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:46.532895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1318/2636] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:46 2022] PUT /identity/v3/projects/8438f3dd16ff433492da5cf51302d0e5/users/5abed076a48b4e239cb9e2021f0960f9/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:46.537116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90503fdf-de47-40ca-a133-c34786d20b43 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:46.547116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90503fdf-de47-40ca-a133-c34786d20b43 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:46.743689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188e62c-8b8e-4b3f-a224-88f3bedbb247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:46.744312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e62c-8b8e-4b3f-a224-88f3bedbb247 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:46.744614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e62c-8b8e-4b3f-a224-88f3bedbb247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:46.755960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e62c-8b8e-4b3f-a224-88f3bedbb247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:46.755960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188e62c-8b8e-4b3f-a224-88f3bedbb247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:46.755960 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188e62c-8b8e-4b3f-a224-88f3bedbb247 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:46.763913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1319/2637] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:16:46.765985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37543f21-0188-4b3e-9f9f-a0763f57caa7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:46.777063 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37543f21-0188-4b3e-9f9f-a0763f57caa7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:46.781750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffa58b-7d68-4aef-86ab-7d27af8ff9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:46.782397 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffa58b-7d68-4aef-86ab-7d27af8ff9e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:46.782674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffa58b-7d68-4aef-86ab-7d27af8ff9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:46.782942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffa58b-7d68-4aef-86ab-7d27af8ff9e3 admin admin] PATH_INFO: `/v3/projects/8438f3dd16ff433492da5cf51302d0e5/users/5abed076a48b4e239cb9e2021f0960f9/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:46.784904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffa58b-7d68-4aef-86ab-7d27af8ff9e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8438f3dd16ff433492da5cf51302d0e5, user_id=5abed076a48b4e239cb9e2021f0960f9, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:46.787522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffa58b-7d68-4aef-86ab-7d27af8ff9e3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:46.808459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1319/2638] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:16:46 2022] PUT /identity/v3/projects/8438f3dd16ff433492da5cf51302d0e5/users/5abed076a48b4e239cb9e2021f0960f9/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:16:46.818974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddf3e4-c253-4aa7-8e68-1fd9ed81dc06 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:46.819453 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddf3e4-c253-4aa7-8e68-1fd9ed81dc06 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:46.819765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddf3e4-c253-4aa7-8e68-1fd9ed81dc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:46.848972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ecddf3e4-c253-4aa7-8e68-1fd9ed81dc06 None None] MFA Rules not processed for user `5abed076a48b4e239cb9e2021f0960f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:16:46.850181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ecddf3e4-c253-4aa7-8e68-1fd9ed81dc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:46.938851 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1320/2639] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:16:46 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:16:46.951083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffa58b-7d68-4aef-86ab-7d27af8ff9e3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:46.952915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a4ffa58b-7d68-4aef-86ab-7d27af8ff9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:46.984649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:47.013234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffa58b-7d68-4aef-86ab-7d27af8ff9e3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:47.016997 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34441b76-2289-45e5-a15e-8ea3200c24e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:47.017756 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34441b76-2289-45e5-a15e-8ea3200c24e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:47.018496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34441b76-2289-45e5-a15e-8ea3200c24e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:47.018496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34441b76-2289-45e5-a15e-8ea3200c24e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:47.028010 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34441b76-2289-45e5-a15e-8ea3200c24e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:47.029555 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34441b76-2289-45e5-a15e-8ea3200c24e2 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:47.049110 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1320/2640] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:46 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:47.419516 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddf3e4-c253-4aa7-8e68-1fd9ed81dc06 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:47.420963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ecddf3e4-c253-4aa7-8e68-1fd9ed81dc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:47.434081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddf3e4-c253-4aa7-8e68-1fd9ed81dc06 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:47.437689 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdde290-ab9f-434a-838f-9ef9108bf587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:47.438283 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdde290-ab9f-434a-838f-9ef9108bf587 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:47.438554 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdde290-ab9f-434a-838f-9ef9108bf587 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:47.438820 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdde290-ab9f-434a-838f-9ef9108bf587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:47.507884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdde290-ab9f-434a-838f-9ef9108bf587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:47.514949 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdde290-ab9f-434a-838f-9ef9108bf587 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:47.532183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1321/2641] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:47 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:48.587721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34441b76-2289-45e5-a15e-8ea3200c24e2 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:48.593072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34441b76-2289-45e5-a15e-8ea3200c24e2 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:48.597928 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67f588-5f2e-438c-9dbf-700068412fbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:48.598571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67f588-5f2e-438c-9dbf-700068412fbe service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:48.598841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67f588-5f2e-438c-9dbf-700068412fbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:48.599103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67f588-5f2e-438c-9dbf-700068412fbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:48.601032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6d67f588-5f2e-438c-9dbf-700068412fbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:48.666767 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67f588-5f2e-438c-9dbf-700068412fbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:48.668307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67f588-5f2e-438c-9dbf-700068412fbe service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:48.677774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1321/2642] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:16:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:16:57.527310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:58.673516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=44 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:16:59.554578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdde290-ab9f-434a-838f-9ef9108bf587 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:59.556198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ebdde290-ab9f-434a-838f-9ef9108bf587 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:16:59.615157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67f588-5f2e-438c-9dbf-700068412fbe service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:16:59.713488 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdde290-ab9f-434a-838f-9ef9108bf587 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:59.718242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:59.718884 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:59.719142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:59.719413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] PATH_INFO: `/v3/users/e078eedf2f094a1e86c4a23b700b04dd` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:59.720591 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e078eedf2f094a1e86c4a23b700b04dd)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:59.721496 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67f588-5f2e-438c-9dbf-700068412fbe service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:16:59.722504 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:59.726716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:16:59.727319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:16:59.727599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:16:59.727867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] PATH_INFO: `/v3/users/7541c41d8d6f473e9c8af7126570569b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:16:59.729036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7541c41d8d6f473e9c8af7126570569b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:16:59.730932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:16:59.980145 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e078eedf2f094a1e86c4a23b700b04dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.002903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7541c41d8d6f473e9c8af7126570569b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.196006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e078eedf2f094a1e86c4a23b700b04dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.210441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e078eedf2f094a1e86c4a23b700b04dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.211192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e078eedf2f094a1e86c4a23b700b04dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.211838 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e078eedf2f094a1e86c4a23b700b04dd'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.229382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1322/2643] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:59 2022] DELETE /identity/v3/users/e078eedf2f094a1e86c4a23b700b04dd => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:00.230353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:00.231666 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:00.250615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:00.292278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7541c41d8d6f473e9c8af7126570569b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.295729 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ac2c2-3dbd-473f-90eb-c96e24b594b0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:00.300081 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7541c41d8d6f473e9c8af7126570569b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.300804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7541c41d8d6f473e9c8af7126570569b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.300936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:00.301709 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:00.301902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-008093f6-9f97-4f6f-93cf-ae8c17e255ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7541c41d8d6f473e9c8af7126570569b'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.302017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:00.302409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] PATH_INFO: `/v3/users/241a3972e0a14223ac8a4ee38996a492` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:00.305309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=241a3972e0a14223ac8a4ee38996a492)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:00.306651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:00.324603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1322/2644] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:16:59 2022] DELETE /identity/v3/users/7541c41d8d6f473e9c8af7126570569b => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:00.326452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e76876-8d85-477a-b940-0cc9262750db None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:00.326730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e76876-8d85-477a-b940-0cc9262750db None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:00.326999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e76876-8d85-477a-b940-0cc9262750db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:00.355312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e0e76876-8d85-477a-b940-0cc9262750db None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:00.356593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e0e76876-8d85-477a-b940-0cc9262750db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:00.399671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:00.414402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1323/2645] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:00.415600 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4bb62-ee62-4613-8002-0118a3f77fde None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:00.415881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4bb62-ee62-4613-8002-0118a3f77fde None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:00.416150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4bb62-ee62-4613-8002-0118a3f77fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:00.434964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9aa4bb62-ee62-4613-8002-0118a3f77fde None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:00.437248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9aa4bb62-ee62-4613-8002-0118a3f77fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:00.444449 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1324/2646] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:00.445245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4bb62-ee62-4613-8002-0118a3f77fde None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:00.446501 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9aa4bb62-ee62-4613-8002-0118a3f77fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:00.520319 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4bb62-ee62-4613-8002-0118a3f77fde None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:00.523980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b0006-0e19-4c89-ab83-c25dd499a4c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:00.524593 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b0006-0e19-4c89-ab83-c25dd499a4c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:00.524861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b0006-0e19-4c89-ab83-c25dd499a4c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:00.525147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b0006-0e19-4c89-ab83-c25dd499a4c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:00.533765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b0006-0e19-4c89-ab83-c25dd499a4c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:00.538022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b0006-0e19-4c89-ab83-c25dd499a4c2 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:00.544259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1325/2647] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:00.545056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b0006-0e19-4c89-ab83-c25dd499a4c2 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:00.546285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b81b0006-0e19-4c89-ab83-c25dd499a4c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:00.557903 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b0006-0e19-4c89-ab83-c25dd499a4c2 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:00.562384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b77e65-fdbc-4b9e-b706-b77425aa043f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:00.566988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b77e65-fdbc-4b9e-b706-b77425aa043f service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:00.566988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b77e65-fdbc-4b9e-b706-b77425aa043f service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:00.566988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b77e65-fdbc-4b9e-b706-b77425aa043f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:00.575035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b77e65-fdbc-4b9e-b706-b77425aa043f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:00.576463 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b77e65-fdbc-4b9e-b706-b77425aa043f service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:00.576633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '241a3972e0a14223ac8a4ee38996a492'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.593163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1326/2648] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:00.647824 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '241a3972e0a14223ac8a4ee38996a492'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.656471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '241a3972e0a14223ac8a4ee38996a492'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.658475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '241a3972e0a14223ac8a4ee38996a492'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.658475 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '241a3972e0a14223ac8a4ee38996a492'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:00.689185 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1323/2649] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:00 2022] DELETE /identity/v3/users/241a3972e0a14223ac8a4ee38996a492 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:00.693570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7f0e4-f156-4ab5-845a-38d28ba764ad None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:00.693861 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7f0e4-f156-4ab5-845a-38d28ba764ad None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:00.694132 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7f0e4-f156-4ab5-845a-38d28ba764ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:00.713816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-2bb7f0e4-f156-4ab5-845a-38d28ba764ad None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:00.714946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2bb7f0e4-f156-4ab5-845a-38d28ba764ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:00.788585 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1327/2650] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:00.808241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:00.808241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:00.883912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0c9ca-bfa3-4208-b8f8-3e103d74fd3f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:00.887663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1faa37-12fa-4bbb-8af3-ec7d4d5ce29a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:00.888318 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1faa37-12fa-4bbb-8af3-ec7d4d5ce29a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:00.888695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1faa37-12fa-4bbb-8af3-ec7d4d5ce29a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:00.889024 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1faa37-12fa-4bbb-8af3-ec7d4d5ce29a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:00.898785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1faa37-12fa-4bbb-8af3-ec7d4d5ce29a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:00.900405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1faa37-12fa-4bbb-8af3-ec7d4d5ce29a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:00.910251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1324/2651] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:01.117586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7f0e4-f156-4ab5-845a-38d28ba764ad None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:01.121921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1faa37-12fa-4bbb-8af3-ec7d4d5ce29a service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:01.122252 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-2bb7f0e4-f156-4ab5-845a-38d28ba764ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:01.123360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5f1faa37-12fa-4bbb-8af3-ec7d4d5ce29a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:01.143303 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7f0e4-f156-4ab5-845a-38d28ba764ad None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:01.145217 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1faa37-12fa-4bbb-8af3-ec7d4d5ce29a service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:01.147877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:01.148570 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:01.149048 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:01.150199 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] PATH_INFO: `/v3/projects/5e8e12824ef24876b78c154ae28a6b44` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:01.151890 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e8e12824ef24876b78c154ae28a6b44)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:01.154022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:01.163233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:01.164064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:01.164856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:01.165189 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] PATH_INFO: `/v3/projects/7541e89df6f84901a968036313b383be` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:01.166238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7541e89df6f84901a968036313b383be)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:01.168133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:01.304691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e8e12824ef24876b78c154ae28a6b44', 'request_id': 'req-516ce601-4197-4287-bb37-810bdd6c989c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.306614 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:01.310682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e8e12824ef24876b78c154ae28a6b44', 'request_id': 'req-516ce601-4197-4287-bb37-810bdd6c989c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.318364 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:01.327660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7541e89df6f84901a968036313b383be', 'request_id': 'req-1195c90d-e731-49a6-a45d-1a3afb7fac00'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.333213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7541e89df6f84901a968036313b383be', 'request_id': 'req-1195c90d-e731-49a6-a45d-1a3afb7fac00'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.419114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e8e12824ef24876b78c154ae28a6b44', 'request_id': 'req-516ce601-4197-4287-bb37-810bdd6c989c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.431589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] The token cache is being invalidate because project 5e8e12824ef24876b78c154ae28a6b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:01.431877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.432532 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.434136 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1328/2652] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:01 2022] DELETE /identity/v3/projects/5e8e12824ef24876b78c154ae28a6b44 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:01.437662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:01.439066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:01.503560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ce601-4197-4287-bb37-810bdd6c989c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:01.509816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7541e89df6f84901a968036313b383be', 'request_id': 'req-1195c90d-e731-49a6-a45d-1a3afb7fac00'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.523704 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] The token cache is being invalidate because project 7541e89df6f84901a968036313b383be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:01.523996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.524763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.526557 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1325/2653] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:01 2022] DELETE /identity/v3/projects/7541e89df6f84901a968036313b383be => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:01.527510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:01.528909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:01.567799 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:01.569141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:01.569279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:01.569583 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] PATH_INFO: `/v3/projects/d3c8986c30914529ac94becd8e0ebfe3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:01.570730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c8986c30914529ac94becd8e0ebfe3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:01.572691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:01.595698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195c90d-e731-49a6-a45d-1a3afb7fac00 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:01.600814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:01.601439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:01.601718 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:01.602109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] PATH_INFO: `/v3/users/87efa71f25d8411098ccd3088f584f4e` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:01.603256 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=87efa71f25d8411098ccd3088f584f4e)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:01.605225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:01.721849 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c8986c30914529ac94becd8e0ebfe3', 'request_id': 'req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.727282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c8986c30914529ac94becd8e0ebfe3', 'request_id': 'req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.829883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c8986c30914529ac94becd8e0ebfe3', 'request_id': 'req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.842493 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] The token cache is being invalidate because project d3c8986c30914529ac94becd8e0ebfe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:01.842777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.843414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.843582 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87efa71f25d8411098ccd3088f584f4e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.845165 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1329/2654] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:01 2022] DELETE /identity/v3/projects/d3c8986c30914529ac94becd8e0ebfe3 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:01.845798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:01.847083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:01.869378 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:01.916032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa9f86-07c4-41cc-b05d-ec5b2c081407 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:01.920828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87efa71f25d8411098ccd3088f584f4e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.921012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:01.921452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:01.921724 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:01.921987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] PATH_INFO: `/v3/users/5abed076a48b4e239cb9e2021f0960f9` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:01.923273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5abed076a48b4e239cb9e2021f0960f9)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:01.925285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:01.932371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87efa71f25d8411098ccd3088f584f4e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.932371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87efa71f25d8411098ccd3088f584f4e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.932371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87efa71f25d8411098ccd3088f584f4e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:01.961170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1326/2655] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:01 2022] DELETE /identity/v3/users/87efa71f25d8411098ccd3088f584f4e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:01.961793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:01.963762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:02.032549 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:02.057077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320eeeb-0b19-46d1-8e48-220c26c082ad admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:02.062663 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:02.063409 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:02.063777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:02.064109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] PATH_INFO: `/v3/users/c4971fa110c141098b912eb8b980dcdb` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:02.065355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4971fa110c141098b912eb8b980dcdb)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:02.067389 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:02.239038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5abed076a48b4e239cb9e2021f0960f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.327722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5abed076a48b4e239cb9e2021f0960f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.335740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5abed076a48b4e239cb9e2021f0960f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.336580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5abed076a48b4e239cb9e2021f0960f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.337916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5abed076a48b4e239cb9e2021f0960f9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.356710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4971fa110c141098b912eb8b980dcdb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.357435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1330/2656] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:01 2022] DELETE /identity/v3/users/5abed076a48b4e239cb9e2021f0960f9 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:02.424265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4971fa110c141098b912eb8b980dcdb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.436761 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4971fa110c141098b912eb8b980dcdb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.436761 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4971fa110c141098b912eb8b980dcdb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.437577 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4971fa110c141098b912eb8b980dcdb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.459750 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1327/2657] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:01 2022] DELETE /identity/v3/users/c4971fa110c141098b912eb8b980dcdb => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:02.560731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:02.562267 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:02.672366 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a6e68-d57e-4b6e-b656-2423b15bbccf admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:02.737902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:02.755545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:02.756224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:02.756512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:02.756796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] PATH_INFO: `/v3/projects/758b38ad039445c7a4dc502283977c0e` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:02.757990 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=758b38ad039445c7a4dc502283977c0e)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:02.760071 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:02.847124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:02.848690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:02.866493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7251-82cb-42cd-be37-2edeebd5816d admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:02.873359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758b38ad039445c7a4dc502283977c0e', 'request_id': 'req-e11e4695-849c-47c6-b118-2bcede6b6b43'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.881862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758b38ad039445c7a4dc502283977c0e', 'request_id': 'req-e11e4695-849c-47c6-b118-2bcede6b6b43'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.936952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:02.937572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:02.937873 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:02.938143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] PATH_INFO: `/v3/projects/8438f3dd16ff433492da5cf51302d0e5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:02.939308 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8438f3dd16ff433492da5cf51302d0e5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:02.941287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:02.961912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758b38ad039445c7a4dc502283977c0e', 'request_id': 'req-e11e4695-849c-47c6-b118-2bcede6b6b43'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.974515 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] The token cache is being invalidate because project 758b38ad039445c7a4dc502283977c0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:02.974952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.975483 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:02.977014 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1331/2658] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:02 2022] DELETE /identity/v3/projects/758b38ad039445c7a4dc502283977c0e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:02.977776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:02.979084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:02.992616 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e4695-849c-47c6-b118-2bcede6b6b43 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:02.997259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:02.997910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:02.998185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:02.998452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] PATH_INFO: `/v3/projects/7fe4857ceae64872ac59d8d5c8fd4f74` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:02.999569 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fe4857ceae64872ac59d8d5c8fd4f74)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.001522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.058752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8438f3dd16ff433492da5cf51302d0e5', 'request_id': 'req-71b8646a-4a65-4771-980c-57d3561dafb5'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.064198 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8438f3dd16ff433492da5cf51302d0e5', 'request_id': 'req-71b8646a-4a65-4771-980c-57d3561dafb5'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.067877 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:03.093944 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fe4857ceae64872ac59d8d5c8fd4f74', 'request_id': 'req-75c166fd-b0d2-42e3-970f-22d91c1c5d17'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.101589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fe4857ceae64872ac59d8d5c8fd4f74', 'request_id': 'req-75c166fd-b0d2-42e3-970f-22d91c1c5d17'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.121065 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8438f3dd16ff433492da5cf51302d0e5', 'request_id': 'req-71b8646a-4a65-4771-980c-57d3561dafb5'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.133743 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] The token cache is being invalidate because project 8438f3dd16ff433492da5cf51302d0e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:03.134025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.134668 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71b8646a-4a65-4771-980c-57d3561dafb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.136186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1328/2659] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:02 2022] DELETE /identity/v3/projects/8438f3dd16ff433492da5cf51302d0e5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:03.139442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fb455-07cc-4550-9245-df9ddd668bf7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.139442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fb455-07cc-4550-9245-df9ddd668bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.139442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fb455-07cc-4550-9245-df9ddd668bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.186906 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-659fb455-07cc-4550-9245-df9ddd668bf7 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:03.188157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-659fb455-07cc-4550-9245-df9ddd668bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:03.240788 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fe4857ceae64872ac59d8d5c8fd4f74', 'request_id': 'req-75c166fd-b0d2-42e3-970f-22d91c1c5d17'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.248589 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1329/2660] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:03.255724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a522e86-ee6d-4ddc-b083-d5617b13cf25 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.255724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a522e86-ee6d-4ddc-b083-d5617b13cf25 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.255724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a522e86-ee6d-4ddc-b083-d5617b13cf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.257367 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] The token cache is being invalidate because project 7fe4857ceae64872ac59d8d5c8fd4f74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:03.257658 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.258379 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-75c166fd-b0d2-42e3-970f-22d91c1c5d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:03.259909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1332/2661] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:02 2022] DELETE /identity/v3/projects/7fe4857ceae64872ac59d8d5c8fd4f74 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:03.261328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa96c80-a0cd-4e5a-a760-54c253a51724 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.261602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa96c80-a0cd-4e5a-a760-54c253a51724 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.262684 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa96c80-a0cd-4e5a-a760-54c253a51724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.273537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-1a522e86-ee6d-4ddc-b083-d5617b13cf25 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:03.275715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1a522e86-ee6d-4ddc-b083-d5617b13cf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:03.282380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6aa96c80-a0cd-4e5a-a760-54c253a51724 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:03.283670 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6aa96c80-a0cd-4e5a-a760-54c253a51724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:03.284008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1330/2662] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:03.285213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e4f81-bad0-4f14-8858-4f7dd41ad47d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.291514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e4f81-bad0-4f14-8858-4f7dd41ad47d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.291514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e4f81-bad0-4f14-8858-4f7dd41ad47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.295495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1333/2663] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:03.295954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa96c80-a0cd-4e5a-a760-54c253a51724 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.305818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa96c80-a0cd-4e5a-a760-54c253a51724 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.306011 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-243e4f81-bad0-4f14-8858-4f7dd41ad47d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:03.307179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-243e4f81-bad0-4f14-8858-4f7dd41ad47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:03.310702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1a5aa-ddf6-4c59-87b1-afdf3452c2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.311283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1a5aa-ddf6-4c59-87b1-afdf3452c2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.311561 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1a5aa-ddf6-4c59-87b1-afdf3452c2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.311821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1a5aa-ddf6-4c59-87b1-afdf3452c2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.312451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1a5aa-ddf6-4c59-87b1-afdf3452c2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:17:03.312780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1a5aa-ddf6-4c59-87b1-afdf3452c2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.314351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1a5aa-ddf6-4c59-87b1-afdf3452c2e5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.314993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1331/2664] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:03.316141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e89c-5708-4aab-8fb2-7549373bf21d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.316418 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e89c-5708-4aab-8fb2-7549373bf21d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.316717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e89c-5708-4aab-8fb2-7549373bf21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.333601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1334/2665] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:17:03.334823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1518a1-6161-468b-a2dd-bd95f48e8986 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.335167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1518a1-6161-468b-a2dd-bd95f48e8986 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.335499 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1518a1-6161-468b-a2dd-bd95f48e8986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.335926 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5563e89c-5708-4aab-8fb2-7549373bf21d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:03.337224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5563e89c-5708-4aab-8fb2-7549373bf21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:03.345074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1332/2666] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:03.345909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563e89c-5708-4aab-8fb2-7549373bf21d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.355099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563e89c-5708-4aab-8fb2-7549373bf21d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.358545 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-8e1518a1-6161-468b-a2dd-bd95f48e8986 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:03.359584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa05379-3936-403c-aede-a89abd5f6372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.359742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8e1518a1-6161-468b-a2dd-bd95f48e8986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:03.360153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa05379-3936-403c-aede-a89abd5f6372 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.360424 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa05379-3936-403c-aede-a89abd5f6372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.360721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa05379-3936-403c-aede-a89abd5f6372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.361320 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa05379-3936-403c-aede-a89abd5f6372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.363173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa05379-3936-403c-aede-a89abd5f6372 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.367908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1335/2667] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:03.368750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1518a1-6161-468b-a2dd-bd95f48e8986 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.378016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1518a1-6161-468b-a2dd-bd95f48e8986 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.383095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2085783-a9d9-4d3b-9db6-daed82d49a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.383657 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2085783-a9d9-4d3b-9db6-daed82d49a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.383922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2085783-a9d9-4d3b-9db6-daed82d49a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.384183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2085783-a9d9-4d3b-9db6-daed82d49a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.384836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2085783-a9d9-4d3b-9db6-daed82d49a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:17:03.385126 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2085783-a9d9-4d3b-9db6-daed82d49a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.386810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2085783-a9d9-4d3b-9db6-daed82d49a54 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.398659 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1336/2668] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:17:03.399359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2085783-a9d9-4d3b-9db6-daed82d49a54 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.408384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2085783-a9d9-4d3b-9db6-daed82d49a54 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.413060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4525659-6253-4258-ba74-c560225b8b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.413594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525659-6253-4258-ba74-c560225b8b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.413862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525659-6253-4258-ba74-c560225b8b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.414123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525659-6253-4258-ba74-c560225b8b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.414714 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4525659-6253-4258-ba74-c560225b8b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:17:03.415000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4525659-6253-4258-ba74-c560225b8b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.416519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4525659-6253-4258-ba74-c560225b8b00 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.427721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1337/2669] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:17:03.429023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4525659-6253-4258-ba74-c560225b8b00 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.434954 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:03.438538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4525659-6253-4258-ba74-c560225b8b00 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.443008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15348e68-8c3e-457a-8d29-fe7444b1e8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.444183 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15348e68-8c3e-457a-8d29-fe7444b1e8ad admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.444451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15348e68-8c3e-457a-8d29-fe7444b1e8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.444759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15348e68-8c3e-457a-8d29-fe7444b1e8ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.445411 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15348e68-8c3e-457a-8d29-fe7444b1e8ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.447188 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15348e68-8c3e-457a-8d29-fe7444b1e8ad admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.493459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1333/2670] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:03.494410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa05379-3936-403c-aede-a89abd5f6372 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.544922 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1338/2671] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:03.546772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15348e68-8c3e-457a-8d29-fe7444b1e8ad admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.559355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa05379-3936-403c-aede-a89abd5f6372 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.589784 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:03.610553 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15348e68-8c3e-457a-8d29-fe7444b1e8ad admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.613231 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655d7b5-21e9-4403-aa41-59765058a08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.613823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655d7b5-21e9-4403-aa41-59765058a08d admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.614101 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655d7b5-21e9-4403-aa41-59765058a08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.614365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655d7b5-21e9-4403-aa41-59765058a08d admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.614981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655d7b5-21e9-4403-aa41-59765058a08d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.616336 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaed3d-eabb-4fe1-a12b-1e60baf4988b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.617037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaed3d-eabb-4fe1-a12b-1e60baf4988b admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.617303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaed3d-eabb-4fe1-a12b-1e60baf4988b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.617560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaed3d-eabb-4fe1-a12b-1e60baf4988b admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.617671 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655d7b5-21e9-4403-aa41-59765058a08d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.618159 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaed3d-eabb-4fe1-a12b-1e60baf4988b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.620029 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaed3d-eabb-4fe1-a12b-1e60baf4988b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.721090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1339/2672] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:03.722320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655d7b5-21e9-4403-aa41-59765058a08d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.722502 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1334/2673] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:03.728583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaed3d-eabb-4fe1-a12b-1e60baf4988b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.733082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655d7b5-21e9-4403-aa41-59765058a08d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.789897 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaed3d-eabb-4fe1-a12b-1e60baf4988b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.794953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004ca4b-4eea-4ce1-b9d7-12a6a8f32b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.795552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004ca4b-4eea-4ce1-b9d7-12a6a8f32b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.795828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004ca4b-4eea-4ce1-b9d7-12a6a8f32b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.796092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004ca4b-4eea-4ce1-b9d7-12a6a8f32b20 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.799931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004ca4b-4eea-4ce1-b9d7-12a6a8f32b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.800086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a9a3e-be06-417f-a5b4-555a29e54883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.800086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a9a3e-be06-417f-a5b4-555a29e54883 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.800086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a9a3e-be06-417f-a5b4-555a29e54883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.800086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a9a3e-be06-417f-a5b4-555a29e54883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.800481 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a9a3e-be06-417f-a5b4-555a29e54883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.800598 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004ca4b-4eea-4ce1-b9d7-12a6a8f32b20 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.801752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a9a3e-be06-417f-a5b4-555a29e54883 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.812653 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1335/2674] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:03.813786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a9a3e-be06-417f-a5b4-555a29e54883 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.823785 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a9a3e-be06-417f-a5b4-555a29e54883 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.828459 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685649ff-262a-445c-bcd9-2c663d21dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.831035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685649ff-262a-445c-bcd9-2c663d21dae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.831035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685649ff-262a-445c-bcd9-2c663d21dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.831035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685649ff-262a-445c-bcd9-2c663d21dae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.831035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685649ff-262a-445c-bcd9-2c663d21dae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.832422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685649ff-262a-445c-bcd9-2c663d21dae1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.863859 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1340/2675] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:03.864962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004ca4b-4eea-4ce1-b9d7-12a6a8f32b20 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.875361 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004ca4b-4eea-4ce1-b9d7-12a6a8f32b20 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.880021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2093904-5d38-4e4e-ac3d-feec362029c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.880610 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2093904-5d38-4e4e-ac3d-feec362029c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.882831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2093904-5d38-4e4e-ac3d-feec362029c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.882831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2093904-5d38-4e4e-ac3d-feec362029c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.882831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2093904-5d38-4e4e-ac3d-feec362029c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.883309 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2093904-5d38-4e4e-ac3d-feec362029c5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.893435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1341/2676] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:03.894195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2093904-5d38-4e4e-ac3d-feec362029c5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.904433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1336/2677] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:03 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:03.905171 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2093904-5d38-4e4e-ac3d-feec362029c5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.906321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685649ff-262a-445c-bcd9-2c663d21dae1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.909867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19501d40-04cc-4655-ace4-7f30e3835fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.910435 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19501d40-04cc-4655-ace4-7f30e3835fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.910709 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19501d40-04cc-4655-ace4-7f30e3835fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.910971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19501d40-04cc-4655-ace4-7f30e3835fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.911587 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19501d40-04cc-4655-ace4-7f30e3835fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.913680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19501d40-04cc-4655-ace4-7f30e3835fe3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.918635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685649ff-262a-445c-bcd9-2c663d21dae1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.923503 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b2ddb-c4f6-488a-a035-450f829c27a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.924150 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b2ddb-c4f6-488a-a035-450f829c27a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.924470 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b2ddb-c4f6-488a-a035-450f829c27a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.924769 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b2ddb-c4f6-488a-a035-450f829c27a9 admin admin] PATH_INFO: `/v3/projects/38f4350420c04537baa6d0311e595540/users/360ff1e563954a06b5ad19b7b9fbe5bc/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.925287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1342/2678] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:03.926117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19501d40-04cc-4655-ace4-7f30e3835fe3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.936968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19501d40-04cc-4655-ace4-7f30e3835fe3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.941559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826522f3-90ca-41ce-816d-0ccd132172aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.946233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826522f3-90ca-41ce-816d-0ccd132172aa admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.946233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826522f3-90ca-41ce-816d-0ccd132172aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.946233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826522f3-90ca-41ce-816d-0ccd132172aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.946233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826522f3-90ca-41ce-816d-0ccd132172aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.946233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826522f3-90ca-41ce-816d-0ccd132172aa admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.946732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b2ddb-c4f6-488a-a035-450f829c27a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f4350420c04537baa6d0311e595540, user_id=360ff1e563954a06b5ad19b7b9fbe5bc, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.947651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b2ddb-c4f6-488a-a035-450f829c27a9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.954839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1343/2679] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:03.955619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826522f3-90ca-41ce-816d-0ccd132172aa admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.965314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826522f3-90ca-41ce-816d-0ccd132172aa admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:03.969911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15192e-0655-427b-a9ec-7629ac73c5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:03.970484 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15192e-0655-427b-a9ec-7629ac73c5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:03.970754 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15192e-0655-427b-a9ec-7629ac73c5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:03.971017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15192e-0655-427b-a9ec-7629ac73c5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:03.971636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15192e-0655-427b-a9ec-7629ac73c5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:03.973281 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15192e-0655-427b-a9ec-7629ac73c5c5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:03.984024 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1344/2680] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:03.987302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15192e-0655-427b-a9ec-7629ac73c5c5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:03.995174 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15192e-0655-427b-a9ec-7629ac73c5c5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.004244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7511b8b-3205-48e4-a950-ae0fefefa3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.004244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7511b8b-3205-48e4-a950-ae0fefefa3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.004244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7511b8b-3205-48e4-a950-ae0fefefa3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.004244 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7511b8b-3205-48e4-a950-ae0fefefa3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.004681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7511b8b-3205-48e4-a950-ae0fefefa3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.004681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7511b8b-3205-48e4-a950-ae0fefefa3d5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.013672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1345/2681] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:04.013672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7511b8b-3205-48e4-a950-ae0fefefa3d5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.024566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7511b8b-3205-48e4-a950-ae0fefefa3d5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.028097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:04.030071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1337/2682] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:03 2022] PUT /identity/v3/projects/38f4350420c04537baa6d0311e595540/users/360ff1e563954a06b5ad19b7b9fbe5bc/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:04.031096 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b2ddb-c4f6-488a-a035-450f829c27a9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.082007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a29ac-e4cc-494a-98f9-7491966f4bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.082595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a29ac-e4cc-494a-98f9-7491966f4bca admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.082872 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a29ac-e4cc-494a-98f9-7491966f4bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.083141 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a29ac-e4cc-494a-98f9-7491966f4bca admin admin] PATH_INFO: `/v3/projects/77e412f2cc2847699fe748946b1b9ce7/users/b6024838c1154d988f85f94f74b11d0f/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.098685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a29ac-e4cc-494a-98f9-7491966f4bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=77e412f2cc2847699fe748946b1b9ce7, user_id=b6024838c1154d988f85f94f74b11d0f, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.102295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a29ac-e4cc-494a-98f9-7491966f4bca admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.108139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b2ddb-c4f6-488a-a035-450f829c27a9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.113171 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fc233-5e96-4de3-a79a-e54b84a3789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.114124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fc233-5e96-4de3-a79a-e54b84a3789e admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.114463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fc233-5e96-4de3-a79a-e54b84a3789e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.114797 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fc233-5e96-4de3-a79a-e54b84a3789e admin admin] PATH_INFO: `/v3/projects/a0b1cc6620654587b42150bb52faccea/users/361e9f17d7f84902b75828b78fb7d4f6/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.129741 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fc233-5e96-4de3-a79a-e54b84a3789e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b1cc6620654587b42150bb52faccea, user_id=361e9f17d7f84902b75828b78fb7d4f6, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.132464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fc233-5e96-4de3-a79a-e54b84a3789e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.135526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:04.213898 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1338/2683] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:04 2022] PUT /identity/v3/projects/a0b1cc6620654587b42150bb52faccea/users/361e9f17d7f84902b75828b78fb7d4f6/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:04.214942 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458f95f-e836-4403-ae8d-9277580f0806 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.215238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458f95f-e836-4403-ae8d-9277580f0806 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.215479 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458f95f-e836-4403-ae8d-9277580f0806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.218692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1346/2684] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:04 2022] PUT /identity/v3/projects/77e412f2cc2847699fe748946b1b9ce7/users/b6024838c1154d988f85f94f74b11d0f/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:04.225548 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e908c-3b08-41b8-a905-7e2374c7dd10 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.225985 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e908c-3b08-41b8-a905-7e2374c7dd10 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.226376 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e908c-3b08-41b8-a905-7e2374c7dd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.245450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-a458f95f-e836-4403-ae8d-9277580f0806 None None] MFA Rules not processed for user `360ff1e563954a06b5ad19b7b9fbe5bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:04.246664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-a458f95f-e836-4403-ae8d-9277580f0806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:04.256263 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-760e908c-3b08-41b8-a905-7e2374c7dd10 None None] MFA Rules not processed for user `361e9f17d7f84902b75828b78fb7d4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:04.257599 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-760e908c-3b08-41b8-a905-7e2374c7dd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:04.341076 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1339/2685] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:04.345853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0742b2d-d73c-4502-819d-0a37c6a15530 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.346209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0742b2d-d73c-4502-819d-0a37c6a15530 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.346487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0742b2d-d73c-4502-819d-0a37c6a15530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.364704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1347/2686] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:04.365072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e908c-3b08-41b8-a905-7e2374c7dd10 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.404344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-d0742b2d-d73c-4502-819d-0a37c6a15530 None None] MFA Rules not processed for user `b6024838c1154d988f85f94f74b11d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:04.405793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d0742b2d-d73c-4502-819d-0a37c6a15530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:04.453062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e908c-3b08-41b8-a905-7e2374c7dd10 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.461898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d623d9-2798-40ce-911f-327a9ca1c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.461898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d623d9-2798-40ce-911f-327a9ca1c051 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.461898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d623d9-2798-40ce-911f-327a9ca1c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.461898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d623d9-2798-40ce-911f-327a9ca1c051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.462283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d623d9-2798-40ce-911f-327a9ca1c051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.462791 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d623d9-2798-40ce-911f-327a9ca1c051 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.479880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1340/2687] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:04.481183 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0742b2d-d73c-4502-819d-0a37c6a15530 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.484332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d0742b2d-d73c-4502-819d-0a37c6a15530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:04.541676 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0742b2d-d73c-4502-819d-0a37c6a15530 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.545371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03257c39-cb78-4c57-b634-16ca07d8f5b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.545972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03257c39-cb78-4c57-b634-16ca07d8f5b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.546302 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03257c39-cb78-4c57-b634-16ca07d8f5b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.546505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03257c39-cb78-4c57-b634-16ca07d8f5b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.554774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03257c39-cb78-4c57-b634-16ca07d8f5b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.556321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03257c39-cb78-4c57-b634-16ca07d8f5b9 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.565074 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1341/2688] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:04.565836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03257c39-cb78-4c57-b634-16ca07d8f5b9 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.569038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1348/2689] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:04 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:04.573045 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d623d9-2798-40ce-911f-327a9ca1c051 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.577332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03257c39-cb78-4c57-b634-16ca07d8f5b9 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.578399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:04.594025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:04.636903 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43eb0f-6912-47ee-b67e-1bf4e36ef8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.637431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43eb0f-6912-47ee-b67e-1bf4e36ef8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.637707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43eb0f-6912-47ee-b67e-1bf4e36ef8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.637975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43eb0f-6912-47ee-b67e-1bf4e36ef8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.638620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43eb0f-6912-47ee-b67e-1bf4e36ef8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.640574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43eb0f-6912-47ee-b67e-1bf4e36ef8e3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.644962 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d623d9-2798-40ce-911f-327a9ca1c051 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.650410 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45a4e3-b29c-4edf-bcf9-bdbf4f47ce9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.651018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45a4e3-b29c-4edf-bcf9-bdbf4f47ce9c admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.651293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45a4e3-b29c-4edf-bcf9-bdbf4f47ce9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.651562 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45a4e3-b29c-4edf-bcf9-bdbf4f47ce9c admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.652310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45a4e3-b29c-4edf-bcf9-bdbf4f47ce9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.654389 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45a4e3-b29c-4edf-bcf9-bdbf4f47ce9c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.729945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1349/2690] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:04 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:04.729945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45a4e3-b29c-4edf-bcf9-bdbf4f47ce9c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.730269 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1342/2691] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:04 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:04.731756 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43eb0f-6912-47ee-b67e-1bf4e36ef8e3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.739909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45a4e3-b29c-4edf-bcf9-bdbf4f47ce9c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.800130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43eb0f-6912-47ee-b67e-1bf4e36ef8e3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.804708 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.805479 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.805651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.805915 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] PATH_INFO: `/v3/users/361e9f17d7f84902b75828b78fb7d4f6` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.806321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e93af-9673-4144-ab18-481178a8c7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.807011 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e93af-9673-4144-ab18-481178a8c7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.807157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=361e9f17d7f84902b75828b78fb7d4f6)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.807410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e93af-9673-4144-ab18-481178a8c7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.807742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e93af-9673-4144-ab18-481178a8c7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.808450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e93af-9673-4144-ab18-481178a8c7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.809181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.810157 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e93af-9673-4144-ab18-481178a8c7d6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.821411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1343/2692] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:04.822444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e93af-9673-4144-ab18-481178a8c7d6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.832717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e93af-9673-4144-ab18-481178a8c7d6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.837624 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707aa2f0-04c1-40ad-894b-0fc0065f2092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.841260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aa2f0-04c1-40ad-894b-0fc0065f2092 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.841260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aa2f0-04c1-40ad-894b-0fc0065f2092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.841260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707aa2f0-04c1-40ad-894b-0fc0065f2092 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.841260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707aa2f0-04c1-40ad-894b-0fc0065f2092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.841260 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707aa2f0-04c1-40ad-894b-0fc0065f2092 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:04.920929 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1344/2693] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:04 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:04.921645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707aa2f0-04c1-40ad-894b-0fc0065f2092 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:04.931436 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707aa2f0-04c1-40ad-894b-0fc0065f2092 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:04.935988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382bf8f-266d-4bd7-acdd-98ac1171896c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:04.936664 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382bf8f-266d-4bd7-acdd-98ac1171896c admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:04.936950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382bf8f-266d-4bd7-acdd-98ac1171896c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:04.937222 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382bf8f-266d-4bd7-acdd-98ac1171896c admin admin] PATH_INFO: `/v3/projects/853e6d32647141a7b89891c88b466423/users/7a24c92b00fb4ec1babed0fdb90e5176/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:04.952159 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382bf8f-266d-4bd7-acdd-98ac1171896c admin admin] RBAC: Authorizing `identity:create_grant(project_id=853e6d32647141a7b89891c88b466423, user_id=7a24c92b00fb4ec1babed0fdb90e5176, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:04.953742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361e9f17d7f84902b75828b78fb7d4f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:04.954881 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382bf8f-266d-4bd7-acdd-98ac1171896c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.000590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361e9f17d7f84902b75828b78fb7d4f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:05.008725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361e9f17d7f84902b75828b78fb7d4f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:05.009365 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361e9f17d7f84902b75828b78fb7d4f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:05.009978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361e9f17d7f84902b75828b78fb7d4f6'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:05.017746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1345/2694] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:04 2022] PUT /identity/v3/projects/853e6d32647141a7b89891c88b466423/users/7a24c92b00fb4ec1babed0fdb90e5176/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:05.019406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382bf8f-266d-4bd7-acdd-98ac1171896c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.022451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6382bf8f-266d-4bd7-acdd-98ac1171896c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.030196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1350/2695] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:04 2022] DELETE /identity/v3/users/361e9f17d7f84902b75828b78fb7d4f6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:05.034033 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.035476 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.049104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:05.083422 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:05.087099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382bf8f-266d-4bd7-acdd-98ac1171896c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.101297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5520bf-4bd2-46fb-abd1-518d73b55df6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.106755 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e3513-645c-41ff-bfd1-890dea499d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.111679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3513-645c-41ff-bfd1-890dea499d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.111679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3513-645c-41ff-bfd1-890dea499d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.111679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3513-645c-41ff-bfd1-890dea499d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.111679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3513-645c-41ff-bfd1-890dea499d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.111679 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3513-645c-41ff-bfd1-890dea499d6b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.120738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1351/2696] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:05.121912 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d97cf1-2087-4c85-8f7e-9103afcca1bd None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.122195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d97cf1-2087-4c85-8f7e-9103afcca1bd None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.122495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d97cf1-2087-4c85-8f7e-9103afcca1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.141283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-08d97cf1-2087-4c85-8f7e-9103afcca1bd None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:05.146139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-08d97cf1-2087-4c85-8f7e-9103afcca1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.146284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afa110-2b43-41d5-8247-6072824b91fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.146284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afa110-2b43-41d5-8247-6072824b91fd admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.146284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afa110-2b43-41d5-8247-6072824b91fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.146284 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afa110-2b43-41d5-8247-6072824b91fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.146712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afa110-2b43-41d5-8247-6072824b91fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.146712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afa110-2b43-41d5-8247-6072824b91fd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.153715 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1352/2697] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:05.153715 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d97cf1-2087-4c85-8f7e-9103afcca1bd None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.159048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1346/2698] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:05.160496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afa110-2b43-41d5-8247-6072824b91fd admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.160919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d97cf1-2087-4c85-8f7e-9103afcca1bd None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.163863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-55afa110-2b43-41d5-8247-6072824b91fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.166739 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492657f-6060-4694-bdf7-aae431030482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.167390 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492657f-6060-4694-bdf7-aae431030482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.167666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492657f-6060-4694-bdf7-aae431030482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.168061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492657f-6060-4694-bdf7-aae431030482 admin admin] PATH_INFO: `/v3/projects/853e6d32647141a7b89891c88b466423/users/7a24c92b00fb4ec1babed0fdb90e5176/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.170038 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492657f-6060-4694-bdf7-aae431030482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=853e6d32647141a7b89891c88b466423, user_id=7a24c92b00fb4ec1babed0fdb90e5176, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.174862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492657f-6060-4694-bdf7-aae431030482 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.220774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afa110-2b43-41d5-8247-6072824b91fd admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.226251 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d93438-99e0-4fa7-8273-cbb44ac614bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.227064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d93438-99e0-4fa7-8273-cbb44ac614bf service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.227413 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d93438-99e0-4fa7-8273-cbb44ac614bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.227686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d93438-99e0-4fa7-8273-cbb44ac614bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.230438 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1353/2699] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:05 2022] PUT /identity/v3/projects/853e6d32647141a7b89891c88b466423/users/7a24c92b00fb4ec1babed0fdb90e5176/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:05.231289 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492657f-6060-4694-bdf7-aae431030482 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.237153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d93438-99e0-4fa7-8273-cbb44ac614bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.238723 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d93438-99e0-4fa7-8273-cbb44ac614bf service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.249445 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1347/2700] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:17:05.250822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a9685-8754-4945-9376-a84706b6f8a7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.251108 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a9685-8754-4945-9376-a84706b6f8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.251374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a9685-8754-4945-9376-a84706b6f8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.281416 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-5d7a9685-8754-4945-9376-a84706b6f8a7 None None] MFA Rules not processed for user `7a24c92b00fb4ec1babed0fdb90e5176`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:05.282629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5d7a9685-8754-4945-9376-a84706b6f8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.300825 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492657f-6060-4694-bdf7-aae431030482 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.306343 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08170611-1ee3-44b8-ac93-bc684a0c926d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.306996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08170611-1ee3-44b8-ac93-bc684a0c926d admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.307272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08170611-1ee3-44b8-ac93-bc684a0c926d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.307538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08170611-1ee3-44b8-ac93-bc684a0c926d admin admin] PATH_INFO: `/v3/projects/71e644ecdce94017a1e2a7e6e07b7be3/users/951f342f57384e51ac7f538433cccb9f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.324863 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08170611-1ee3-44b8-ac93-bc684a0c926d admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e644ecdce94017a1e2a7e6e07b7be3, user_id=951f342f57384e51ac7f538433cccb9f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.327793 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08170611-1ee3-44b8-ac93-bc684a0c926d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.358583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1348/2701] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:05.369692 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a9685-8754-4945-9376-a84706b6f8a7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.371032 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5d7a9685-8754-4945-9376-a84706b6f8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.400764 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:05.425017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a9685-8754-4945-9376-a84706b6f8a7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.429197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8611f-9d86-45e2-9ef5-8482bfc61ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.432167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8611f-9d86-45e2-9ef5-8482bfc61ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.432167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8611f-9d86-45e2-9ef5-8482bfc61ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.432167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8611f-9d86-45e2-9ef5-8482bfc61ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.438844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8611f-9d86-45e2-9ef5-8482bfc61ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.440311 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8611f-9d86-45e2-9ef5-8482bfc61ef6 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.449395 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1349/2702] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:17:05.457667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1354/2703] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:05 2022] PUT /identity/v3/projects/71e644ecdce94017a1e2a7e6e07b7be3/users/951f342f57384e51ac7f538433cccb9f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:05.461392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8611f-9d86-45e2-9ef5-8482bfc61ef6 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.473788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8611f-9d86-45e2-9ef5-8482bfc61ef6 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.574481 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08170611-1ee3-44b8-ac93-bc684a0c926d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.579214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308be10-b823-40fe-b214-7182fe0ce2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.580279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308be10-b823-40fe-b214-7182fe0ce2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.580599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308be10-b823-40fe-b214-7182fe0ce2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.580875 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308be10-b823-40fe-b214-7182fe0ce2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.581963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308be10-b823-40fe-b214-7182fe0ce2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.585541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308be10-b823-40fe-b214-7182fe0ce2b9 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.596939 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1350/2704] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:05.601679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308be10-b823-40fe-b214-7182fe0ce2b9 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.614290 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308be10-b823-40fe-b214-7182fe0ce2b9 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.620996 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ef4b2-cb47-4e9b-86e5-2a8341c0f4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.621661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ef4b2-cb47-4e9b-86e5-2a8341c0f4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.622099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ef4b2-cb47-4e9b-86e5-2a8341c0f4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.622597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ef4b2-cb47-4e9b-86e5-2a8341c0f4fd admin admin] PATH_INFO: `/v3/projects/71e644ecdce94017a1e2a7e6e07b7be3/users/951f342f57384e51ac7f538433cccb9f/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.625468 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ef4b2-cb47-4e9b-86e5-2a8341c0f4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e644ecdce94017a1e2a7e6e07b7be3, user_id=951f342f57384e51ac7f538433cccb9f, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.628122 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ef4b2-cb47-4e9b-86e5-2a8341c0f4fd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.644585 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1351/2705] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:05 2022] PUT /identity/v3/projects/71e644ecdce94017a1e2a7e6e07b7be3/users/951f342f57384e51ac7f538433cccb9f/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:05.654009 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440660-2bbb-41bb-95b5-c160a25c6dea None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.656813 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440660-2bbb-41bb-95b5-c160a25c6dea None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.657094 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440660-2bbb-41bb-95b5-c160a25c6dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.658804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08170611-1ee3-44b8-ac93-bc684a0c926d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.694582 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:05.703120 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-7d440660-2bbb-41bb-95b5-c160a25c6dea None None] MFA Rules not processed for user `951f342f57384e51ac7f538433cccb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:05.704348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7d440660-2bbb-41bb-95b5-c160a25c6dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.720016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.720659 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.720939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.721213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.723612 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.785710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1352/2706] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:05.786605 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440660-2bbb-41bb-95b5-c160a25c6dea None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.790268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7d440660-2bbb-41bb-95b5-c160a25c6dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.791521 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.794622 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.806167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1355/2707] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:05.807351 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:05.816998 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:05.840373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3ccc0-7764-4cf9-bc54-9a923006c22d service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.845575 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f053-2d92-4d37-9428-1254615111d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.846302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f053-2d92-4d37-9428-1254615111d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.846672 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f053-2d92-4d37-9428-1254615111d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.847518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f053-2d92-4d37-9428-1254615111d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.855793 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:05.867510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47f053-2d92-4d37-9428-1254615111d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.867510 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47f053-2d92-4d37-9428-1254615111d3 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:05.875277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1356/2708] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:05.880458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440660-2bbb-41bb-95b5-c160a25c6dea None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:05.886206 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:05.887004 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:05.887280 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:05.887550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] PATH_INFO: `/v3/projects/a0b1cc6620654587b42150bb52faccea` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:05.888736 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b1cc6620654587b42150bb52faccea)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:05.890715 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:06.026952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b1cc6620654587b42150bb52faccea', 'request_id': 'req-508b05a8-8c94-472b-b5db-162782bde762'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:06.030809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b1cc6620654587b42150bb52faccea', 'request_id': 'req-508b05a8-8c94-472b-b5db-162782bde762'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:06.115465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b1cc6620654587b42150bb52faccea', 'request_id': 'req-508b05a8-8c94-472b-b5db-162782bde762'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:06.127233 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] The token cache is being invalidate because project a0b1cc6620654587b42150bb52faccea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:06.127517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:06.128142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:06.129675 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1353/2709] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:05 2022] DELETE /identity/v3/projects/a0b1cc6620654587b42150bb52faccea => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:06.135265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce79fc1-805f-4ba0-b967-58a2cbe9ad73 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.135796 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce79fc1-805f-4ba0-b967-58a2cbe9ad73 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.135908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce79fc1-805f-4ba0-b967-58a2cbe9ad73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.165304 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-5ce79fc1-805f-4ba0-b967-58a2cbe9ad73 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:06.166888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-5ce79fc1-805f-4ba0-b967-58a2cbe9ad73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:06.167412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:06.168880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:06.228593 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b05a8-8c94-472b-b5db-162782bde762 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:06.232229 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d63e95-1830-4d40-99d9-27894afa69b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:06.232789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1357/2710] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:06.232932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d63e95-1830-4d40-99d9-27894afa69b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.233262 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d63e95-1830-4d40-99d9-27894afa69b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.233568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d63e95-1830-4d40-99d9-27894afa69b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.235349 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-04d63e95-1830-4d40-99d9-27894afa69b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:06.241350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c512e0-7b98-404f-96e6-b290658bf1d4 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.241632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c512e0-7b98-404f-96e6-b290658bf1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.241896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c512e0-7b98-404f-96e6-b290658bf1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.265698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e3c512e0-7b98-404f-96e6-b290658bf1d4 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:06.266984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e3c512e0-7b98-404f-96e6-b290658bf1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:06.280265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1358/2711] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:06.285295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c512e0-7b98-404f-96e6-b290658bf1d4 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:06.293317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c512e0-7b98-404f-96e6-b290658bf1d4 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:06.296803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d63e95-1830-4d40-99d9-27894afa69b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:06.297982 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0e9db-0f3a-4482-9d01-20921d49b7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:06.298239 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d63e95-1830-4d40-99d9-27894afa69b6 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:06.298559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0e9db-0f3a-4482-9d01-20921d49b7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.298832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0e9db-0f3a-4482-9d01-20921d49b7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.299095 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0e9db-0f3a-4482-9d01-20921d49b7ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.299733 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0e9db-0f3a-4482-9d01-20921d49b7ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:17:06.300057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0e9db-0f3a-4482-9d01-20921d49b7ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:06.301655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0e9db-0f3a-4482-9d01-20921d49b7ce admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:06.308377 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1354/2712] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:17:06.317320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1359/2713] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:17:06.318269 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d63e95-1830-4d40-99d9-27894afa69b6 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:06.329522 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d63e95-1830-4d40-99d9-27894afa69b6 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:06.334507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62263d-2fe9-4cc8-b92e-483d9774837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:06.335093 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62263d-2fe9-4cc8-b92e-483d9774837e admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.335368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62263d-2fe9-4cc8-b92e-483d9774837e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.335635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62263d-2fe9-4cc8-b92e-483d9774837e admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.336257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62263d-2fe9-4cc8-b92e-483d9774837e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:06.338411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62263d-2fe9-4cc8-b92e-483d9774837e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:06.470827 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1355/2714] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:06 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:17:06.475133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0e9db-0f3a-4482-9d01-20921d49b7ce admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:06.485283 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0e9db-0f3a-4482-9d01-20921d49b7ce admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:06.532627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=100 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:06.542491 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137df7a-b958-4d9d-ae52-3da46c9741ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:06.543039 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137df7a-b958-4d9d-ae52-3da46c9741ba admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.543310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137df7a-b958-4d9d-ae52-3da46c9741ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.543710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137df7a-b958-4d9d-ae52-3da46c9741ba admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.544187 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137df7a-b958-4d9d-ae52-3da46c9741ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:06.546265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137df7a-b958-4d9d-ae52-3da46c9741ba admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:06.630945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1360/2715] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:06 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:17:06.635684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62263d-2fe9-4cc8-b92e-483d9774837e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:06.642694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:06.649293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62263d-2fe9-4cc8-b92e-483d9774837e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:06.654316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939fa03-3340-4f21-bc8e-40f4d5b45269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:06.654908 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939fa03-3340-4f21-bc8e-40f4d5b45269 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.655178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939fa03-3340-4f21-bc8e-40f4d5b45269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.655465 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939fa03-3340-4f21-bc8e-40f4d5b45269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.656095 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939fa03-3340-4f21-bc8e-40f4d5b45269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:06.658086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939fa03-3340-4f21-bc8e-40f4d5b45269 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:06.670288 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1356/2716] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:17:06.674181 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137df7a-b958-4d9d-ae52-3da46c9741ba admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:06.685120 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137df7a-b958-4d9d-ae52-3da46c9741ba admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:06.690895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dd66a-ee8b-4150-afa1-d5febbc6de30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:06.691629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd66a-ee8b-4150-afa1-d5febbc6de30 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.691759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd66a-ee8b-4150-afa1-d5febbc6de30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.692015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd66a-ee8b-4150-afa1-d5febbc6de30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.692765 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dd66a-ee8b-4150-afa1-d5febbc6de30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:06.694592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dd66a-ee8b-4150-afa1-d5febbc6de30 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:06.711063 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939fa03-3340-4f21-bc8e-40f4d5b45269 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:06.712227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1361/2717] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:06.722385 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939fa03-3340-4f21-bc8e-40f4d5b45269 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:06.727888 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abcd213-fe86-4b61-b013-1786b681f83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:06.728583 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcd213-fe86-4b61-b013-1786b681f83e admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.728859 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcd213-fe86-4b61-b013-1786b681f83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.729128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcd213-fe86-4b61-b013-1786b681f83e admin admin] PATH_INFO: `/v3/projects/9e956c74d0ec4521b0a792e0e242b695/users/4d85e61124af4c69919f33e647e14c75/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.745423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abcd213-fe86-4b61-b013-1786b681f83e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e956c74d0ec4521b0a792e0e242b695, user_id=4d85e61124af4c69919f33e647e14c75, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:06.749324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abcd213-fe86-4b61-b013-1786b681f83e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:06.846552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1357/2718] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:06 2022] PUT /identity/v3/projects/9e956c74d0ec4521b0a792e0e242b695/users/4d85e61124af4c69919f33e647e14c75/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:06.852344 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283d993-deb5-4048-9dcc-e49fba8785bf None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:06.852665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283d993-deb5-4048-9dcc-e49fba8785bf None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:06.852960 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283d993-deb5-4048-9dcc-e49fba8785bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:06.875909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-9283d993-deb5-4048-9dcc-e49fba8785bf None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:06.877150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9283d993-deb5-4048-9dcc-e49fba8785bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:06.945312 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1362/2719] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:06.955173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abcd213-fe86-4b61-b013-1786b681f83e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:06.964682 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7abcd213-fe86-4b61-b013-1786b681f83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:07.032227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abcd213-fe86-4b61-b013-1786b681f83e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:07.038611 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d3025-9066-4120-84ba-26281e0bcfc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:07.039155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d3025-9066-4120-84ba-26281e0bcfc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:07.039443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d3025-9066-4120-84ba-26281e0bcfc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:07.039693 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d3025-9066-4120-84ba-26281e0bcfc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:07.051848 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d3025-9066-4120-84ba-26281e0bcfc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:07.053578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d3025-9066-4120-84ba-26281e0bcfc3 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:07.066116 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1358/2720] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:07.371717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283d993-deb5-4048-9dcc-e49fba8785bf None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:07.382627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9283d993-deb5-4048-9dcc-e49fba8785bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:07.386007 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283d993-deb5-4048-9dcc-e49fba8785bf None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:07.389581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a0356-8813-4120-9102-34c373128cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:07.390193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a0356-8813-4120-9102-34c373128cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:07.390448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a0356-8813-4120-9102-34c373128cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:07.390710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a0356-8813-4120-9102-34c373128cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:07.392321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-716a0356-8813-4120-9102-34c373128cb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:07.464445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a0356-8813-4120-9102-34c373128cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:07.465905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a0356-8813-4120-9102-34c373128cb1 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:07.475620 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1363/2721] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:17:08.531054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b213f9-9269-4ea3-8561-184db76c5745 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:08.531569 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b213f9-9269-4ea3-8561-184db76c5745 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:08.531696 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b213f9-9269-4ea3-8561-184db76c5745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:08.560133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-66b213f9-9269-4ea3-8561-184db76c5745 None None] MFA Rules not processed for user `4d85e61124af4c69919f33e647e14c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:08.561392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-66b213f9-9269-4ea3-8561-184db76c5745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:08.631006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1359/2722] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:08.642911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a0356-8813-4120-9102-34c373128cb1 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:08.698266 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a0356-8813-4120-9102-34c373128cb1 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:08.702051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6450af-3cea-4946-a95f-1fa3c38f1e36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:08.702647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6450af-3cea-4946-a95f-1fa3c38f1e36 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:08.702920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6450af-3cea-4946-a95f-1fa3c38f1e36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:08.703185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6450af-3cea-4946-a95f-1fa3c38f1e36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:08.717377 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6450af-3cea-4946-a95f-1fa3c38f1e36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:08.719522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6450af-3cea-4946-a95f-1fa3c38f1e36 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:08.729375 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1364/2723] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:09.064426 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b213f9-9269-4ea3-8561-184db76c5745 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:09.066530 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-66b213f9-9269-4ea3-8561-184db76c5745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:09.082155 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b213f9-9269-4ea3-8561-184db76c5745 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:09.086943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:09.087562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:09.087840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:09.088111 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] PATH_INFO: `/v3/users/360ff1e563954a06b5ad19b7b9fbe5bc` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:09.089266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=360ff1e563954a06b5ad19b7b9fbe5bc)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:09.092514 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:09.299928 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '360ff1e563954a06b5ad19b7b9fbe5bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:09.447815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '360ff1e563954a06b5ad19b7b9fbe5bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:09.452732 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:09.460803 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '360ff1e563954a06b5ad19b7b9fbe5bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:09.461592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '360ff1e563954a06b5ad19b7b9fbe5bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:09.462345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '360ff1e563954a06b5ad19b7b9fbe5bc'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:09.493864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e78cc-e30d-43f9-9cfa-1de4d286a43a None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:09.494393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e78cc-e30d-43f9-9cfa-1de4d286a43a None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:09.494931 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e78cc-e30d-43f9-9cfa-1de4d286a43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:09.498031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1360/2724] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:09 2022] DELETE /identity/v3/users/360ff1e563954a06b5ad19b7b9fbe5bc => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:09.511092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:09.517646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7d1e78cc-e30d-43f9-9cfa-1de4d286a43a None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:09.518913 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7d1e78cc-e30d-43f9-9cfa-1de4d286a43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:09.581984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1365/2725] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:09.594361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:09.595927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:09.661225 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef45310-1899-440f-a047-e69a9ee5cf59 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:09.665440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd172abb-aa6d-4423-94c2-b60d0921bd3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:09.666055 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd172abb-aa6d-4423-94c2-b60d0921bd3a service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:09.666335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd172abb-aa6d-4423-94c2-b60d0921bd3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:09.666602 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd172abb-aa6d-4423-94c2-b60d0921bd3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:09.676652 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd172abb-aa6d-4423-94c2-b60d0921bd3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:09.678700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd172abb-aa6d-4423-94c2-b60d0921bd3a service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:09.689254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1361/2726] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:10.186994 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e78cc-e30d-43f9-9cfa-1de4d286a43a None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:10.188716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7d1e78cc-e30d-43f9-9cfa-1de4d286a43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:10.225518 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e78cc-e30d-43f9-9cfa-1de4d286a43a None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:10.237368 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:10.240635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:10.240920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:10.241190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] PATH_INFO: `/v3/projects/38f4350420c04537baa6d0311e595540` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:10.242920 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f4350420c04537baa6d0311e595540)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:10.244899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:10.426797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f4350420c04537baa6d0311e595540', 'request_id': 'req-91203599-56ec-4e75-8275-b5b02332fd3f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.433148 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f4350420c04537baa6d0311e595540', 'request_id': 'req-91203599-56ec-4e75-8275-b5b02332fd3f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.543182 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f4350420c04537baa6d0311e595540', 'request_id': 'req-91203599-56ec-4e75-8275-b5b02332fd3f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.572150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] The token cache is being invalidate because project 38f4350420c04537baa6d0311e595540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:10.573503 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.577861 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a1b50-2dea-45c7-860a-625c81b6f3eb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:10.578141 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a1b50-2dea-45c7-860a-625c81b6f3eb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:10.578430 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a1b50-2dea-45c7-860a-625c81b6f3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:10.580560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.583427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1366/2727] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:10 2022] DELETE /identity/v3/projects/38f4350420c04537baa6d0311e595540 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:10.588297 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:10.589740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:10.613850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-229a1b50-2dea-45c7-860a-625c81b6f3eb None None] MFA Rules not processed for user `4d85e61124af4c69919f33e647e14c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:10.617355 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-229a1b50-2dea-45c7-860a-625c81b6f3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:10.674083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1362/2728] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:10.674345 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203599-56ec-4e75-8275-b5b02332fd3f admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:10.679105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:10.683976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:10.683976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:10.683976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] PATH_INFO: `/v3/users/7a24c92b00fb4ec1babed0fdb90e5176` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:10.683976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a24c92b00fb4ec1babed0fdb90e5176)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:10.683976 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:10.871152 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:10.884945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a24c92b00fb4ec1babed0fdb90e5176'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.967789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a24c92b00fb4ec1babed0fdb90e5176'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.977285 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a24c92b00fb4ec1babed0fdb90e5176'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.978123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a24c92b00fb4ec1babed0fdb90e5176'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.978750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-85062f3a-a5cb-400a-9c71-33ad1521c6f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a24c92b00fb4ec1babed0fdb90e5176'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:10.997676 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1367/2729] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:10 2022] DELETE /identity/v3/users/7a24c92b00fb4ec1babed0fdb90e5176 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:11.413361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a1b50-2dea-45c7-860a-625c81b6f3eb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:11.424066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a1b50-2dea-45c7-860a-625c81b6f3eb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:11.481673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:11.482319 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:11.482588 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:11.482856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] PATH_INFO: `/v3/projects/853e6d32647141a7b89891c88b466423` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:11.483949 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=853e6d32647141a7b89891c88b466423)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:11.486021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:11.510869 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:11.608014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '853e6d32647141a7b89891c88b466423', 'request_id': 'req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:11.615843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '853e6d32647141a7b89891c88b466423', 'request_id': 'req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:11.732254 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '853e6d32647141a7b89891c88b466423', 'request_id': 'req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:11.744081 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] The token cache is being invalidate because project 853e6d32647141a7b89891c88b466423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:11.746369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:11.746369 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-33a3a7e2-4eb5-4e55-8d94-4e94ce9fc029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:11.746587 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1363/2730] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:11 2022] DELETE /identity/v3/projects/853e6d32647141a7b89891c88b466423 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:11.751530 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a779c3-4893-4b85-b05b-12b5ed44eb55 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:11.752005 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a779c3-4893-4b85-b05b-12b5ed44eb55 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:11.752103 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a779c3-4893-4b85-b05b-12b5ed44eb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:11.771327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-b5a779c3-4893-4b85-b05b-12b5ed44eb55 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:11.773036 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b5a779c3-4893-4b85-b05b-12b5ed44eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:11.835189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1368/2731] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:11.843048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d5bfd-ef4f-4ebf-b221-1b32200f3a11 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:11.843332 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d5bfd-ef4f-4ebf-b221-1b32200f3a11 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:11.843597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d5bfd-ef4f-4ebf-b221-1b32200f3a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:11.866435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3c4d5bfd-ef4f-4ebf-b221-1b32200f3a11 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:11.867678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3c4d5bfd-ef4f-4ebf-b221-1b32200f3a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:11.875520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1364/2732] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:11.879748 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a779c3-4893-4b85-b05b-12b5ed44eb55 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:11.890022 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a779c3-4893-4b85-b05b-12b5ed44eb55 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:11.894691 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204c704-85ad-4493-b3d1-2e395cd9d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:11.895296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204c704-85ad-4493-b3d1-2e395cd9d938 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:11.895563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204c704-85ad-4493-b3d1-2e395cd9d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:11.895919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204c704-85ad-4493-b3d1-2e395cd9d938 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:11.897224 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204c704-85ad-4493-b3d1-2e395cd9d938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:17:11.897517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204c704-85ad-4493-b3d1-2e395cd9d938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:11.899084 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204c704-85ad-4493-b3d1-2e395cd9d938 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:11.910702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1369/2733] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:17:11.914851 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d5bfd-ef4f-4ebf-b221-1b32200f3a11 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:11.923844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d5bfd-ef4f-4ebf-b221-1b32200f3a11 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:11.928425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a69f2-dee0-49d7-8130-575d742d67f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:11.929031 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a69f2-dee0-49d7-8130-575d742d67f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:11.929301 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a69f2-dee0-49d7-8130-575d742d67f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:11.929559 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a69f2-dee0-49d7-8130-575d742d67f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:11.930172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a69f2-dee0-49d7-8130-575d742d67f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:11.932082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a69f2-dee0-49d7-8130-575d742d67f7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:12.025765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1365/2734] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:11 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:12.029867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204c704-85ad-4493-b3d1-2e395cd9d938 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:12.039855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204c704-85ad-4493-b3d1-2e395cd9d938 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:12.096062 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8a139-e8f3-44e7-93ca-6dec635b0666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:12.096706 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8a139-e8f3-44e7-93ca-6dec635b0666 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:12.097050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8a139-e8f3-44e7-93ca-6dec635b0666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:12.097242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8a139-e8f3-44e7-93ca-6dec635b0666 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:12.097845 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8a139-e8f3-44e7-93ca-6dec635b0666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:12.099750 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8a139-e8f3-44e7-93ca-6dec635b0666 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:12.235621 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1370/2735] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:12 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:12.239913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a69f2-dee0-49d7-8130-575d742d67f7 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:12.250343 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a69f2-dee0-49d7-8130-575d742d67f7 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:12.254986 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea005d8-f5f6-49e2-9cd6-dbe36a552319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:12.255568 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea005d8-f5f6-49e2-9cd6-dbe36a552319 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:12.256948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea005d8-f5f6-49e2-9cd6-dbe36a552319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:12.256948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea005d8-f5f6-49e2-9cd6-dbe36a552319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:12.257435 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea005d8-f5f6-49e2-9cd6-dbe36a552319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:12.259008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea005d8-f5f6-49e2-9cd6-dbe36a552319 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:12.268901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1366/2736] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:12.273574 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8a139-e8f3-44e7-93ca-6dec635b0666 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:12.283452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8a139-e8f3-44e7-93ca-6dec635b0666 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:12.287992 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f588c20-2b19-4bd6-996b-756f5288ea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:12.288663 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f588c20-2b19-4bd6-996b-756f5288ea82 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:12.288940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f588c20-2b19-4bd6-996b-756f5288ea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:12.289200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f588c20-2b19-4bd6-996b-756f5288ea82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:12.289819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f588c20-2b19-4bd6-996b-756f5288ea82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:12.291388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f588c20-2b19-4bd6-996b-756f5288ea82 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:12.301674 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1371/2737] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:12.306870 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea005d8-f5f6-49e2-9cd6-dbe36a552319 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:12.318017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea005d8-f5f6-49e2-9cd6-dbe36a552319 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:12.324072 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f99ff-9dce-44d0-9c5e-aa3cef73d4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:12.324973 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f99ff-9dce-44d0-9c5e-aa3cef73d4c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:12.325249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f99ff-9dce-44d0-9c5e-aa3cef73d4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:12.325521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f99ff-9dce-44d0-9c5e-aa3cef73d4c6 admin admin] PATH_INFO: `/v3/projects/4c683c2c4df24fabb08a088b2f0c54f5/users/c08015e100734cfaa6366afc3df766fc/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:12.353680 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f99ff-9dce-44d0-9c5e-aa3cef73d4c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c683c2c4df24fabb08a088b2f0c54f5, user_id=c08015e100734cfaa6366afc3df766fc, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:12.356721 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f99ff-9dce-44d0-9c5e-aa3cef73d4c6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:12.435249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1367/2738] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:12 2022] PUT /identity/v3/projects/4c683c2c4df24fabb08a088b2f0c54f5/users/c08015e100734cfaa6366afc3df766fc/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:12.447097 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfa91c-7f7a-46b7-9e1d-d312d9399a4e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:12.447382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfa91c-7f7a-46b7-9e1d-d312d9399a4e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:12.447649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfa91c-7f7a-46b7-9e1d-d312d9399a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:12.476115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-53dfa91c-7f7a-46b7-9e1d-d312d9399a4e None None] MFA Rules not processed for user `c08015e100734cfaa6366afc3df766fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:12.477552 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-53dfa91c-7f7a-46b7-9e1d-d312d9399a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:12.481949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:12.543789 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1372/2739] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:12.548441 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f99ff-9dce-44d0-9c5e-aa3cef73d4c6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:12.609633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f99ff-9dce-44d0-9c5e-aa3cef73d4c6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:12.614352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212e08c-a95f-4bc7-8f8c-a61ddb6ca714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:12.614950 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212e08c-a95f-4bc7-8f8c-a61ddb6ca714 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:12.615760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212e08c-a95f-4bc7-8f8c-a61ddb6ca714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:12.616026 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212e08c-a95f-4bc7-8f8c-a61ddb6ca714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:12.627353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212e08c-a95f-4bc7-8f8c-a61ddb6ca714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:12.627353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212e08c-a95f-4bc7-8f8c-a61ddb6ca714 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:12.705361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1368/2740] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:12 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:12.709500 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfa91c-7f7a-46b7-9e1d-d312d9399a4e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:12.720617 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfa91c-7f7a-46b7-9e1d-d312d9399a4e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:12.775835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd8d30-061c-4fb6-a79c-5f3e6cb2a7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:12.776578 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd8d30-061c-4fb6-a79c-5f3e6cb2a7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:12.776702 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd8d30-061c-4fb6-a79c-5f3e6cb2a7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:12.776959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd8d30-061c-4fb6-a79c-5f3e6cb2a7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:12.777571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd8d30-061c-4fb6-a79c-5f3e6cb2a7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:12.779490 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd8d30-061c-4fb6-a79c-5f3e6cb2a7f8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:12.876114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1373/2741] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:12 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:17:12.889705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212e08c-a95f-4bc7-8f8c-a61ddb6ca714 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:12.895352 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212e08c-a95f-4bc7-8f8c-a61ddb6ca714 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:12.900480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97362083-1276-4eb1-9666-8370e058bc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:12.901319 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97362083-1276-4eb1-9666-8370e058bc47 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:12.901599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97362083-1276-4eb1-9666-8370e058bc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:12.901867 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97362083-1276-4eb1-9666-8370e058bc47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:12.902513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97362083-1276-4eb1-9666-8370e058bc47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:12.904105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97362083-1276-4eb1-9666-8370e058bc47 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:12.915864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1369/2742] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:12.920807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd8d30-061c-4fb6-a79c-5f3e6cb2a7f8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:12.931904 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd8d30-061c-4fb6-a79c-5f3e6cb2a7f8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:12.936883 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110791e-1490-4327-b06b-7393f3dda333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:12.937533 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110791e-1490-4327-b06b-7393f3dda333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:12.937810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110791e-1490-4327-b06b-7393f3dda333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:12.938082 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110791e-1490-4327-b06b-7393f3dda333 admin admin] PATH_INFO: `/v3/projects/c7cd01ee605b41818d3c82cc62dc9e78/users/4bdfb4dd300c4f498ec4ca1232d063c4/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:12.953096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110791e-1490-4327-b06b-7393f3dda333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cd01ee605b41818d3c82cc62dc9e78, user_id=4bdfb4dd300c4f498ec4ca1232d063c4, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:12.955769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110791e-1490-4327-b06b-7393f3dda333 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:13.076586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1374/2743] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:12 2022] PUT /identity/v3/projects/c7cd01ee605b41818d3c82cc62dc9e78/users/4bdfb4dd300c4f498ec4ca1232d063c4/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:13.080537 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97362083-1276-4eb1-9666-8370e058bc47 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:13.093191 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97362083-1276-4eb1-9666-8370e058bc47 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:13.111296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:13.152948 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781a58a-3823-4999-a753-1dd27760a035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:13.153883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781a58a-3823-4999-a753-1dd27760a035 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:13.154137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781a58a-3823-4999-a753-1dd27760a035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:13.154402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781a58a-3823-4999-a753-1dd27760a035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:13.155042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781a58a-3823-4999-a753-1dd27760a035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:13.156691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781a58a-3823-4999-a753-1dd27760a035 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:13.167822 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1370/2744] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:17:13.172251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110791e-1490-4327-b06b-7393f3dda333 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:13.182630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110791e-1490-4327-b06b-7393f3dda333 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:13.187666 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4c93f-836f-4d1c-b491-208402063cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:13.188315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4c93f-836f-4d1c-b491-208402063cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:13.188623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4c93f-836f-4d1c-b491-208402063cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:13.188896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4c93f-836f-4d1c-b491-208402063cd1 admin admin] PATH_INFO: `/v3/projects/c7cd01ee605b41818d3c82cc62dc9e78/users/4bdfb4dd300c4f498ec4ca1232d063c4/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:13.190870 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4c93f-836f-4d1c-b491-208402063cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cd01ee605b41818d3c82cc62dc9e78, user_id=4bdfb4dd300c4f498ec4ca1232d063c4, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:13.193705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4c93f-836f-4d1c-b491-208402063cd1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:13.204041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1375/2745] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:13 2022] PUT /identity/v3/projects/c7cd01ee605b41818d3c82cc62dc9e78/users/4bdfb4dd300c4f498ec4ca1232d063c4/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:13.214951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d280c-332a-4fda-b5de-722586cd0ddb None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:13.215236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d280c-332a-4fda-b5de-722586cd0ddb None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:13.215640 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d280c-332a-4fda-b5de-722586cd0ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:13.238665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4c93f-836f-4d1c-b491-208402063cd1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:13.240367 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d8c4c93f-836f-4d1c-b491-208402063cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:13.246379 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-125d280c-332a-4fda-b5de-722586cd0ddb None None] MFA Rules not processed for user `4bdfb4dd300c4f498ec4ca1232d063c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:13.247572 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-125d280c-332a-4fda-b5de-722586cd0ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:13.302369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:13.306200 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4c93f-836f-4d1c-b491-208402063cd1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:13.311134 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:13.311752 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:13.312029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:13.312292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] PATH_INFO: `/v3/users/b6024838c1154d988f85f94f74b11d0f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:13.313590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6024838c1154d988f85f94f74b11d0f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:13.315487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:13.318460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1371/2746] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:13.329661 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d280c-332a-4fda-b5de-722586cd0ddb None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:13.331218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-125d280c-332a-4fda-b5de-722586cd0ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:13.390054 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d280c-332a-4fda-b5de-722586cd0ddb None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:13.393734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fbccb-95e6-4b48-8c04-64e6c443388c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:13.394324 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fbccb-95e6-4b48-8c04-64e6c443388c service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:13.394599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fbccb-95e6-4b48-8c04-64e6c443388c service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:13.394865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fbccb-95e6-4b48-8c04-64e6c443388c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:13.449921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:13.452335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fbccb-95e6-4b48-8c04-64e6c443388c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:13.453808 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fbccb-95e6-4b48-8c04-64e6c443388c service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:13.463599 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1372/2747] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:17:13.497172 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6024838c1154d988f85f94f74b11d0f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:13.572564 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6024838c1154d988f85f94f74b11d0f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:13.580391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6024838c1154d988f85f94f74b11d0f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:13.581558 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6024838c1154d988f85f94f74b11d0f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:13.582372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6024838c1154d988f85f94f74b11d0f'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:13.601315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1376/2748] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:13 2022] DELETE /identity/v3/users/b6024838c1154d988f85f94f74b11d0f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:13.605777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ee697-e760-48e9-888f-448efdfe3f1d None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:13.606090 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ee697-e760-48e9-888f-448efdfe3f1d None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:13.606367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ee697-e760-48e9-888f-448efdfe3f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:13.624997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-3c2ee697-e760-48e9-888f-448efdfe3f1d None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:13.626197 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3c2ee697-e760-48e9-888f-448efdfe3f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:13.685698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1373/2749] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:13.695877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:13.697305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:13.756681 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cd5ca-3297-4d80-b360-17e51734471b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:13.761080 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b0a9-1a5c-4cec-b619-648dc31d2d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:13.761899 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b0a9-1a5c-4cec-b619-648dc31d2d74 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:13.768409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b0a9-1a5c-4cec-b619-648dc31d2d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:13.768409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b0a9-1a5c-4cec-b619-648dc31d2d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:13.773334 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921b0a9-1a5c-4cec-b619-648dc31d2d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:13.774636 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921b0a9-1a5c-4cec-b619-648dc31d2d74 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:13.785193 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1377/2750] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:13.896085 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ee697-e760-48e9-888f-448efdfe3f1d None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:13.897578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3c2ee697-e760-48e9-888f-448efdfe3f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:13.988614 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ee697-e760-48e9-888f-448efdfe3f1d None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:13.994788 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aefb3-a00b-455c-8d2a-8158c5085f03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:13.995439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aefb3-a00b-455c-8d2a-8158c5085f03 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:13.995722 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aefb3-a00b-455c-8d2a-8158c5085f03 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:13.995988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aefb3-a00b-455c-8d2a-8158c5085f03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:13.998057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-d18aefb3-a00b-455c-8d2a-8158c5085f03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:14.011252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18aefb3-a00b-455c-8d2a-8158c5085f03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:14.012955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18aefb3-a00b-455c-8d2a-8158c5085f03 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:14.024347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1374/2751] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:14.226355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b0a9-1a5c-4cec-b619-648dc31d2d74 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:14.227984 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3921b0a9-1a5c-4cec-b619-648dc31d2d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:14.249504 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b0a9-1a5c-4cec-b619-648dc31d2d74 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:14.263826 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:14.264460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:14.264764 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:14.265030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] PATH_INFO: `/v3/projects/77e412f2cc2847699fe748946b1b9ce7` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:14.266158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77e412f2cc2847699fe748946b1b9ce7)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:14.268104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:14.379737 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77e412f2cc2847699fe748946b1b9ce7', 'request_id': 'req-cb852453-40ae-445a-b7cb-00c50bdd20b0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.387094 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77e412f2cc2847699fe748946b1b9ce7', 'request_id': 'req-cb852453-40ae-445a-b7cb-00c50bdd20b0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.452705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77e412f2cc2847699fe748946b1b9ce7', 'request_id': 'req-cb852453-40ae-445a-b7cb-00c50bdd20b0'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.465467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] The token cache is being invalidate because project 77e412f2cc2847699fe748946b1b9ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:14.465834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.466469 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.468317 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1378/2752] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:14 2022] DELETE /identity/v3/projects/77e412f2cc2847699fe748946b1b9ce7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:14.472106 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aefb3-a00b-455c-8d2a-8158c5085f03 service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:14.482432 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aefb3-a00b-455c-8d2a-8158c5085f03 service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:14.540286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:14.541066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:14.541344 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:14.541609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] PATH_INFO: `/v3/users/951f342f57384e51ac7f538433cccb9f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:14.542895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] RBAC: Authorizing `identity:delete_user(user_id=951f342f57384e51ac7f538433cccb9f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:14.544955 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:14.552370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:14.739789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '951f342f57384e51ac7f538433cccb9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.846618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '951f342f57384e51ac7f538433cccb9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.854546 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '951f342f57384e51ac7f538433cccb9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.855203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '951f342f57384e51ac7f538433cccb9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.855801 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-53126a9b-6d34-4376-bc8b-69543d85188c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '951f342f57384e51ac7f538433cccb9f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:14.874143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1375/2753] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:14 2022] DELETE /identity/v3/users/951f342f57384e51ac7f538433cccb9f => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:15.432363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:15.434632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:15.444731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:15.451018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb852453-40ae-445a-b7cb-00c50bdd20b0 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:15.518238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:15.518780 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:15.519051 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:15.519310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] PATH_INFO: `/v3/projects/71e644ecdce94017a1e2a7e6e07b7be3` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:15.520662 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e644ecdce94017a1e2a7e6e07b7be3)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:15.523741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:15.609623 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e644ecdce94017a1e2a7e6e07b7be3', 'request_id': 'req-8be2a624-06a9-4518-b4e5-a8972bc9fce2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:15.616215 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e644ecdce94017a1e2a7e6e07b7be3', 'request_id': 'req-8be2a624-06a9-4518-b4e5-a8972bc9fce2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:15.704949 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e644ecdce94017a1e2a7e6e07b7be3', 'request_id': 'req-8be2a624-06a9-4518-b4e5-a8972bc9fce2'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:15.719680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] The token cache is being invalidate because project 71e644ecdce94017a1e2a7e6e07b7be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:15.719964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:15.720647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8be2a624-06a9-4518-b4e5-a8972bc9fce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:15.722131 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1379/2754] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:15 2022] DELETE /identity/v3/projects/71e644ecdce94017a1e2a7e6e07b7be3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:15.726984 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02dec6-2049-4753-bbe3-169a7c9ad6da None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:15.727855 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02dec6-2049-4753-bbe3-169a7c9ad6da None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:15.728338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02dec6-2049-4753-bbe3-169a7c9ad6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:15.747831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2a02dec6-2049-4753-bbe3-169a7c9ad6da None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:15.749086 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2a02dec6-2049-4753-bbe3-169a7c9ad6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:15.813099 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1376/2755] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:15.822156 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdf997-b505-4762-8b12-1a175e235845 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:15.822434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdf997-b505-4762-8b12-1a175e235845 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:15.822699 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdf997-b505-4762-8b12-1a175e235845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:15.843338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-1efdf997-b505-4762-8b12-1a175e235845 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:15.844571 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1efdf997-b505-4762-8b12-1a175e235845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:15.853515 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1380/2756] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:15.859293 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02dec6-2049-4753-bbe3-169a7c9ad6da None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:15.870553 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02dec6-2049-4753-bbe3-169a7c9ad6da None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:15.875449 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a6ff2-03b3-41d6-af98-105fa87b593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:15.876050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a6ff2-03b3-41d6-af98-105fa87b593f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:15.876323 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a6ff2-03b3-41d6-af98-105fa87b593f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:15.876616 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a6ff2-03b3-41d6-af98-105fa87b593f admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:15.877327 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a6ff2-03b3-41d6-af98-105fa87b593f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:17:15.877620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a6ff2-03b3-41d6-af98-105fa87b593f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:15.879200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a6ff2-03b3-41d6-af98-105fa87b593f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:15.891597 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1377/2757] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:17:15.896012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdf997-b505-4762-8b12-1a175e235845 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:15.913000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdf997-b505-4762-8b12-1a175e235845 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:15.918018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6d72f-1cca-40d1-a0a3-51685e7fe5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:15.919066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6d72f-1cca-40d1-a0a3-51685e7fe5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:15.919340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6d72f-1cca-40d1-a0a3-51685e7fe5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:15.919603 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6d72f-1cca-40d1-a0a3-51685e7fe5c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:15.920218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6d72f-1cca-40d1-a0a3-51685e7fe5c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:15.922149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6d72f-1cca-40d1-a0a3-51685e7fe5c8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:16.036650 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1381/2758] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:15 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:16.041615 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a6ff2-03b3-41d6-af98-105fa87b593f admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:16.055136 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a6ff2-03b3-41d6-af98-105fa87b593f admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:16.113023 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de8cb7-7e9c-4bc1-b3bb-ad35490940be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:16.113630 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de8cb7-7e9c-4bc1-b3bb-ad35490940be admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:16.113907 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de8cb7-7e9c-4bc1-b3bb-ad35490940be admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:16.114175 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de8cb7-7e9c-4bc1-b3bb-ad35490940be admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:16.114796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de8cb7-7e9c-4bc1-b3bb-ad35490940be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:16.117257 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de8cb7-7e9c-4bc1-b3bb-ad35490940be admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:16.275961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1378/2759] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:16 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:16.280115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6d72f-1cca-40d1-a0a3-51685e7fe5c8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:16.290611 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6d72f-1cca-40d1-a0a3-51685e7fe5c8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:16.295388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab21f2-22cb-4e25-9beb-43b86d116967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:16.296563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab21f2-22cb-4e25-9beb-43b86d116967 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:16.296563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab21f2-22cb-4e25-9beb-43b86d116967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:16.296563 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab21f2-22cb-4e25-9beb-43b86d116967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:16.297362 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab21f2-22cb-4e25-9beb-43b86d116967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:16.298965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab21f2-22cb-4e25-9beb-43b86d116967 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:16.310077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1382/2760] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:17:16.314030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de8cb7-7e9c-4bc1-b3bb-ad35490940be admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:16.319901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:16.325338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de8cb7-7e9c-4bc1-b3bb-ad35490940be admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:16.330100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d19d8-f075-4480-b976-fc2b86c6341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:16.330778 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d19d8-f075-4480-b976-fc2b86c6341f admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:16.331049 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d19d8-f075-4480-b976-fc2b86c6341f admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:16.331575 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d19d8-f075-4480-b976-fc2b86c6341f admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:16.332235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d19d8-f075-4480-b976-fc2b86c6341f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:16.333856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d19d8-f075-4480-b976-fc2b86c6341f admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:16.344800 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1379/2761] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:16.348969 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab21f2-22cb-4e25-9beb-43b86d116967 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:16.358919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab21f2-22cb-4e25-9beb-43b86d116967 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:16.363680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fc851-99cd-4e0b-b430-7f1204f8598d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:16.364328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fc851-99cd-4e0b-b430-7f1204f8598d admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:16.364633 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fc851-99cd-4e0b-b430-7f1204f8598d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:16.365647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fc851-99cd-4e0b-b430-7f1204f8598d admin admin] PATH_INFO: `/v3/projects/e2dcc28aa1924987b1af652e08971f54/users/99ecf612a52a4a2b84d8cb3284bf6060/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:16.382716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9fc851-99cd-4e0b-b430-7f1204f8598d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2dcc28aa1924987b1af652e08971f54, user_id=99ecf612a52a4a2b84d8cb3284bf6060, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:16.385471 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9fc851-99cd-4e0b-b430-7f1204f8598d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:16.486878 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1383/2762] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:16 2022] PUT /identity/v3/projects/e2dcc28aa1924987b1af652e08971f54/users/99ecf612a52a4a2b84d8cb3284bf6060/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:16.553509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf7ac-8663-4522-8e17-8e2187159626 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:16.553807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf7ac-8663-4522-8e17-8e2187159626 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:16.554131 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf7ac-8663-4522-8e17-8e2187159626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:16.588752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-c84bf7ac-8663-4522-8e17-8e2187159626 None None] MFA Rules not processed for user `99ecf612a52a4a2b84d8cb3284bf6060`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:16.590028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c84bf7ac-8663-4522-8e17-8e2187159626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:16.594527 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fc851-99cd-4e0b-b430-7f1204f8598d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:16.596692 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ee9fc851-99cd-4e0b-b430-7f1204f8598d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:16.635001 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:16.668471 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1380/2763] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:16.674037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf7ac-8663-4522-8e17-8e2187159626 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:16.680957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fc851-99cd-4e0b-b430-7f1204f8598d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:16.684794 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf7ac-8663-4522-8e17-8e2187159626 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:16.686705 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:16.687439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:16.688011 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:16.688213 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] PATH_INFO: `/v3/users/c08015e100734cfaa6366afc3df766fc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:16.689489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08015e100734cfaa6366afc3df766fc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:16.689748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cea927-e32e-4106-8a10-08ab046d34ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:16.690339 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cea927-e32e-4106-8a10-08ab046d34ef admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:16.690600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cea927-e32e-4106-8a10-08ab046d34ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:16.690868 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cea927-e32e-4106-8a10-08ab046d34ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:16.691493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cea927-e32e-4106-8a10-08ab046d34ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:16.691940 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:16.693455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cea927-e32e-4106-8a10-08ab046d34ef admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:16.814958 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1381/2764] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:16 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:17:16.819025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cea927-e32e-4106-8a10-08ab046d34ef admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:16.891410 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cea927-e32e-4106-8a10-08ab046d34ef admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:16.896219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42cc54-7c24-47bc-89ff-87194f5994ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:16.897438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42cc54-7c24-47bc-89ff-87194f5994ea admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:16.897787 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42cc54-7c24-47bc-89ff-87194f5994ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:16.898053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42cc54-7c24-47bc-89ff-87194f5994ea admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:16.898691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42cc54-7c24-47bc-89ff-87194f5994ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:16.900618 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42cc54-7c24-47bc-89ff-87194f5994ea admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:16.926819 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08015e100734cfaa6366afc3df766fc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:17.001405 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1382/2765] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:16 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:17:17.005363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42cc54-7c24-47bc-89ff-87194f5994ea admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:17.015296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42cc54-7c24-47bc-89ff-87194f5994ea admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:17.018893 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08015e100734cfaa6366afc3df766fc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:17.020865 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc9b1-52e8-4bd7-8e9f-546bf49be7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:17.028883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc9b1-52e8-4bd7-8e9f-546bf49be7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:17.028883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc9b1-52e8-4bd7-8e9f-546bf49be7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:17.028883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc9b1-52e8-4bd7-8e9f-546bf49be7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:17.028883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fc9b1-52e8-4bd7-8e9f-546bf49be7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:17.028883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fc9b1-52e8-4bd7-8e9f-546bf49be7fe admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:17.032726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08015e100734cfaa6366afc3df766fc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:17.033580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08015e100734cfaa6366afc3df766fc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:17.034468 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-a5a747da-63ed-455b-a8ac-dcaa61c75071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08015e100734cfaa6366afc3df766fc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:17.035401 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1383/2766] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:17:17.040221 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc9b1-52e8-4bd7-8e9f-546bf49be7fe admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:17.041987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-3e4fc9b1-52e8-4bd7-8e9f-546bf49be7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:17.056167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc9b1-52e8-4bd7-8e9f-546bf49be7fe admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:17.061180 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1384/2767] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:16 2022] DELETE /identity/v3/users/c08015e100734cfaa6366afc3df766fc => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:17.062580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bed3b7-3a5f-4775-8a41-1b4a3dd08bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:17.062726 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0bf57-7d31-46c8-ba53-2edd62529d83 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:17.062818 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bed3b7-3a5f-4775-8a41-1b4a3dd08bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:17.062942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0bf57-7d31-46c8-ba53-2edd62529d83 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:17.063032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0bf57-7d31-46c8-ba53-2edd62529d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:17.063249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bed3b7-3a5f-4775-8a41-1b4a3dd08bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:17.063423 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bed3b7-3a5f-4775-8a41-1b4a3dd08bb1 admin admin] PATH_INFO: `/v3/projects/ba845c07da8e438f8b1d6e877f9c2b02/users/db3eb5f2b0ed4c10a0b3632325b6b675/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:17.080539 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bed3b7-3a5f-4775-8a41-1b4a3dd08bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba845c07da8e438f8b1d6e877f9c2b02, user_id=db3eb5f2b0ed4c10a0b3632325b6b675, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:17.083251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e3c0bf57-7d31-46c8-ba53-2edd62529d83 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:17.083535 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bed3b7-3a5f-4775-8a41-1b4a3dd08bb1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:17.084972 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e3c0bf57-7d31-46c8-ba53-2edd62529d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:17.149061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1385/2768] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:17.152114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1384/2769] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:17 2022] PUT /identity/v3/projects/ba845c07da8e438f8b1d6e877f9c2b02/users/db3eb5f2b0ed4c10a0b3632325b6b675/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:17.157546 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0bf57-7d31-46c8-ba53-2edd62529d83 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:17.163586 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bed3b7-3a5f-4775-8a41-1b4a3dd08bb1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:17.165388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-88bed3b7-3a5f-4775-8a41-1b4a3dd08bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:17.169218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0bf57-7d31-46c8-ba53-2edd62529d83 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:17.185821 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:17.226847 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:17.241027 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7096b-4ee5-40af-8669-2f4d08e18ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:17.241778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7096b-4ee5-40af-8669-2f4d08e18ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:17.242049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7096b-4ee5-40af-8669-2f4d08e18ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:17.242310 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7096b-4ee5-40af-8669-2f4d08e18ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:17.243009 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7096b-4ee5-40af-8669-2f4d08e18ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:17.244865 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7096b-4ee5-40af-8669-2f4d08e18ccc admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:17.252724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bed3b7-3a5f-4775-8a41-1b4a3dd08bb1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:17.256675 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1386/2770] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:17.260549 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7096b-4ee5-40af-8669-2f4d08e18ccc admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:17.260705 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8593ced-efa8-43e8-bd8f-d199e48bbc75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:17.261275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8593ced-efa8-43e8-bd8f-d199e48bbc75 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:17.261555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8593ced-efa8-43e8-bd8f-d199e48bbc75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:17.261825 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8593ced-efa8-43e8-bd8f-d199e48bbc75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:17.272213 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8593ced-efa8-43e8-bd8f-d199e48bbc75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:17.272933 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7096b-4ee5-40af-8669-2f4d08e18ccc admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:17.274932 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8593ced-efa8-43e8-bd8f-d199e48bbc75 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:17.278142 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beeecf-e26e-41d3-8cf0-6756b2f836f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:17.279000 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beeecf-e26e-41d3-8cf0-6756b2f836f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:17.279800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beeecf-e26e-41d3-8cf0-6756b2f836f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:17.279800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beeecf-e26e-41d3-8cf0-6756b2f836f6 admin admin] PATH_INFO: `/v3/projects/ba845c07da8e438f8b1d6e877f9c2b02/users/db3eb5f2b0ed4c10a0b3632325b6b675/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:17.282088 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beeecf-e26e-41d3-8cf0-6756b2f836f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba845c07da8e438f8b1d6e877f9c2b02, user_id=db3eb5f2b0ed4c10a0b3632325b6b675, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:17.284698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1385/2771] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:17:17.285072 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beeecf-e26e-41d3-8cf0-6756b2f836f6 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:17.296688 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1387/2772] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:17 2022] PUT /identity/v3/projects/ba845c07da8e438f8b1d6e877f9c2b02/users/db3eb5f2b0ed4c10a0b3632325b6b675/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:17.307452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8517e50-b704-443f-a5ac-33ba55cc89b3 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:17.307864 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8517e50-b704-443f-a5ac-33ba55cc89b3 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:17.308368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8517e50-b704-443f-a5ac-33ba55cc89b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:17.339802 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e8517e50-b704-443f-a5ac-33ba55cc89b3 None None] MFA Rules not processed for user `db3eb5f2b0ed4c10a0b3632325b6b675`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:17.341310 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e8517e50-b704-443f-a5ac-33ba55cc89b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:17.417983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1386/2773] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:17.428941 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beeecf-e26e-41d3-8cf0-6756b2f836f6 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:17.430627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a7beeecf-e26e-41d3-8cf0-6756b2f836f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:17.489364 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beeecf-e26e-41d3-8cf0-6756b2f836f6 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:17.494273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697a5b8-0cb1-4fbe-9304-33ce81af2a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:17.494951 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697a5b8-0cb1-4fbe-9304-33ce81af2a33 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:17.495226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697a5b8-0cb1-4fbe-9304-33ce81af2a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:17.495495 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697a5b8-0cb1-4fbe-9304-33ce81af2a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:17.505357 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697a5b8-0cb1-4fbe-9304-33ce81af2a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:17.507414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697a5b8-0cb1-4fbe-9304-33ce81af2a33 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:17.516958 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1388/2774] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:17.734574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8517e50-b704-443f-a5ac-33ba55cc89b3 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:17.737236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e8517e50-b704-443f-a5ac-33ba55cc89b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:17.824275 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8517e50-b704-443f-a5ac-33ba55cc89b3 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:17.830555 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:17.831266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:17.831857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:17.832194 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] PATH_INFO: `/v3/projects/4c683c2c4df24fabb08a088b2f0c54f5` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:17.833654 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c683c2c4df24fabb08a088b2f0c54f5)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:17.835579 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:17.864092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:17.971273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c683c2c4df24fabb08a088b2f0c54f5', 'request_id': 'req-8c3e5ef4-b8d5-4581-9408-047509128feb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:17.978199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c683c2c4df24fabb08a088b2f0c54f5', 'request_id': 'req-8c3e5ef4-b8d5-4581-9408-047509128feb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:18.060641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c683c2c4df24fabb08a088b2f0c54f5', 'request_id': 'req-8c3e5ef4-b8d5-4581-9408-047509128feb'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:18.073192 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] The token cache is being invalidate because project 4c683c2c4df24fabb08a088b2f0c54f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:18.073485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:18.074681 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:18.076400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1387/2775] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:17 2022] DELETE /identity/v3/projects/4c683c2c4df24fabb08a088b2f0c54f5 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:18.081099 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697a5b8-0cb1-4fbe-9304-33ce81af2a33 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:18.093025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697a5b8-0cb1-4fbe-9304-33ce81af2a33 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:18.162996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:18.163626 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:18.163906 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:18.164170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] PATH_INFO: `/v3/users/4bdfb4dd300c4f498ec4ca1232d063c4` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:18.165349 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdfb4dd300c4f498ec4ca1232d063c4)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:18.167298 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:18.392242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdfb4dd300c4f498ec4ca1232d063c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:18.460434 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdfb4dd300c4f498ec4ca1232d063c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:18.469911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdfb4dd300c4f498ec4ca1232d063c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:18.470646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdfb4dd300c4f498ec4ca1232d063c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:18.471338 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdfb4dd300c4f498ec4ca1232d063c4'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:18.483751 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:18.492242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1389/2776] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:18 2022] DELETE /identity/v3/users/4bdfb4dd300c4f498ec4ca1232d063c4 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:18.744490 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:18.745913 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:18.760053 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e5ef4-b8d5-4581-9408-047509128feb admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:18.824545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:18.828365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:18.828365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:18.828365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] PATH_INFO: `/v3/users/99ecf612a52a4a2b84d8cb3284bf6060` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:18.828365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99ecf612a52a4a2b84d8cb3284bf6060)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:18.829567 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:18.933683 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:18.935076 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:18.948428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17995c8-e875-48f2-bf8b-c0d95c252d8a admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:18.953402 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:18.954040 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:18.954320 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:18.954589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] PATH_INFO: `/v3/projects/c7cd01ee605b41818d3c82cc62dc9e78` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:18.955731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7cd01ee605b41818d3c82cc62dc9e78)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:18.958311 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:18.997167 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99ecf612a52a4a2b84d8cb3284bf6060'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.121050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99ecf612a52a4a2b84d8cb3284bf6060'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.129814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99ecf612a52a4a2b84d8cb3284bf6060'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.130495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99ecf612a52a4a2b84d8cb3284bf6060'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.131804 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b627e60b-a3db-48b9-960b-240cdf0a4213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99ecf612a52a4a2b84d8cb3284bf6060'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.138924 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7cd01ee605b41818d3c82cc62dc9e78', 'request_id': 'req-6142de00-8c49-4332-9635-f984c2946cfd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.144743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7cd01ee605b41818d3c82cc62dc9e78', 'request_id': 'req-6142de00-8c49-4332-9635-f984c2946cfd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.149594 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1388/2777] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:18 2022] DELETE /identity/v3/users/99ecf612a52a4a2b84d8cb3284bf6060 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:19.153952 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968a6d5-29b3-4f46-8460-f05237bbce1b None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.154235 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968a6d5-29b3-4f46-8460-f05237bbce1b None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.154505 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968a6d5-29b3-4f46-8460-f05237bbce1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.176005 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b968a6d5-29b3-4f46-8460-f05237bbce1b None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:19.176467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b968a6d5-29b3-4f46-8460-f05237bbce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:19.203752 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:19.229725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7cd01ee605b41818d3c82cc62dc9e78', 'request_id': 'req-6142de00-8c49-4332-9635-f984c2946cfd'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.243286 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] The token cache is being invalidate because project c7cd01ee605b41818d3c82cc62dc9e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:19.248641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.248641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6142de00-8c49-4332-9635-f984c2946cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.248641 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1390/2778] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:18 2022] DELETE /identity/v3/projects/c7cd01ee605b41818d3c82cc62dc9e78 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:19.248982 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1389/2779] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:19.251204 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f429ed-2f05-45e7-9735-2058191035f2 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.251487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f429ed-2f05-45e7-9735-2058191035f2 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.251757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f429ed-2f05-45e7-9735-2058191035f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.264351 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968a6d5-29b3-4f46-8460-f05237bbce1b None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:19.265753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b968a6d5-29b3-4f46-8460-f05237bbce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:19.274059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-74f429ed-2f05-45e7-9735-2058191035f2 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:19.275253 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-74f429ed-2f05-45e7-9735-2058191035f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:19.285654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1391/2780] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:19.293060 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b080e-5448-4670-9693-8c493bc90edc None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.293339 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b080e-5448-4670-9693-8c493bc90edc None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.293608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b080e-5448-4670-9693-8c493bc90edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.314226 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-877b080e-5448-4670-9693-8c493bc90edc None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:19.315427 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-877b080e-5448-4670-9693-8c493bc90edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:19.324489 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1392/2781] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:19.328651 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b080e-5448-4670-9693-8c493bc90edc None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:19.336480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968a6d5-29b3-4f46-8460-f05237bbce1b None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:19.340157 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b080e-5448-4670-9693-8c493bc90edc None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:19.340981 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfe4ca-8eaa-4c19-92fd-9ef92f9fb844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:19.341739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfe4ca-8eaa-4c19-92fd-9ef92f9fb844 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.342067 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfe4ca-8eaa-4c19-92fd-9ef92f9fb844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.342496 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfe4ca-8eaa-4c19-92fd-9ef92f9fb844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.344427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1acfe4ca-8eaa-4c19-92fd-9ef92f9fb844 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:19.345196 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992564b5-85c3-46ac-b726-6e57cf132981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:19.345778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992564b5-85c3-46ac-b726-6e57cf132981 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.346049 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992564b5-85c3-46ac-b726-6e57cf132981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.346315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992564b5-85c3-46ac-b726-6e57cf132981 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.346978 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992564b5-85c3-46ac-b726-6e57cf132981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:17:19.347271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992564b5-85c3-46ac-b726-6e57cf132981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:19.348913 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992564b5-85c3-46ac-b726-6e57cf132981 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:19.356398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfe4ca-8eaa-4c19-92fd-9ef92f9fb844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:19.357843 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfe4ca-8eaa-4c19-92fd-9ef92f9fb844 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:19.360406 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1393/2782] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 18:17:19.364678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992564b5-85c3-46ac-b726-6e57cf132981 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:19.368028 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1390/2783] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:17:19.374589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992564b5-85c3-46ac-b726-6e57cf132981 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:19.379179 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad222c87-79c6-4bb9-a231-5c380f90403b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:19.379757 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad222c87-79c6-4bb9-a231-5c380f90403b admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.380029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad222c87-79c6-4bb9-a231-5c380f90403b admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.380293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad222c87-79c6-4bb9-a231-5c380f90403b admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.381506 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad222c87-79c6-4bb9-a231-5c380f90403b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:19.383383 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad222c87-79c6-4bb9-a231-5c380f90403b admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:19.522577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1394/2784] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:19 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:17:19.526100 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfe4ca-8eaa-4c19-92fd-9ef92f9fb844 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:19.538077 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfe4ca-8eaa-4c19-92fd-9ef92f9fb844 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:19.598216 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39b843-130a-473a-a244-920811f16de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:19.598934 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39b843-130a-473a-a244-920811f16de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.599218 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39b843-130a-473a-a244-920811f16de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.599975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39b843-130a-473a-a244-920811f16de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.600765 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39b843-130a-473a-a244-920811f16de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:19.602840 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39b843-130a-473a-a244-920811f16de0 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:19.681882 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1391/2785] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:19 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:17:19.686154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad222c87-79c6-4bb9-a231-5c380f90403b admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:19.696464 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad222c87-79c6-4bb9-a231-5c380f90403b admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:19.701041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262a57c-f3ff-447e-8459-c37f86bcdbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:19.701625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262a57c-f3ff-447e-8459-c37f86bcdbec admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.701894 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262a57c-f3ff-447e-8459-c37f86bcdbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.702160 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262a57c-f3ff-447e-8459-c37f86bcdbec admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.702835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262a57c-f3ff-447e-8459-c37f86bcdbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:19.704451 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262a57c-f3ff-447e-8459-c37f86bcdbec admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:19.717628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1395/2786] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:19.718836 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39b843-130a-473a-a244-920811f16de0 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:19.729287 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39b843-130a-473a-a244-920811f16de0 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:19.733900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab4814-a447-4358-a5a5-34ce4927a8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:19.734467 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab4814-a447-4358-a5a5-34ce4927a8fe admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.734738 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab4814-a447-4358-a5a5-34ce4927a8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.735000 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab4814-a447-4358-a5a5-34ce4927a8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.735625 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab4814-a447-4358-a5a5-34ce4927a8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:19.737279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab4814-a447-4358-a5a5-34ce4927a8fe admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:19.747766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1392/2787] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:17:19.751742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262a57c-f3ff-447e-8459-c37f86bcdbec admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:19.761921 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262a57c-f3ff-447e-8459-c37f86bcdbec admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:19.766613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2a177-33da-4126-a2e9-26890b415fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:19.773630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2a177-33da-4126-a2e9-26890b415fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.773630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2a177-33da-4126-a2e9-26890b415fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.773630 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2a177-33da-4126-a2e9-26890b415fa5 admin admin] PATH_INFO: `/v3/projects/8727724683894b19af25c3889e4dad00/users/f1f63c450ef64400b86bee86b63bc200/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.776380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab4814-a447-4358-a5a5-34ce4927a8fe admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:19.778282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-5dab4814-a447-4358-a5a5-34ce4927a8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:19.784741 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2a177-33da-4126-a2e9-26890b415fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8727724683894b19af25c3889e4dad00, user_id=f1f63c450ef64400b86bee86b63bc200, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:19.789352 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2a177-33da-4126-a2e9-26890b415fa5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:19.799604 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab4814-a447-4358-a5a5-34ce4927a8fe admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:19.807033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:19.807645 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.807909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.808172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] PATH_INFO: `/v3/projects/e2dcc28aa1924987b1af652e08971f54` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.809434 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2dcc28aa1924987b1af652e08971f54)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:19.811338 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:19.881645 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1396/2788] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:19 2022] PUT /identity/v3/projects/8727724683894b19af25c3889e4dad00/users/f1f63c450ef64400b86bee86b63bc200/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:19.924526 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:19.926620 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2dcc28aa1924987b1af652e08971f54', 'request_id': 'req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.932879 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2dcc28aa1924987b1af652e08971f54', 'request_id': 'req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:19.950371 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f067b-7fec-4603-bb45-274d353d0c60 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:19.950697 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f067b-7fec-4603-bb45-274d353d0c60 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:19.950952 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f067b-7fec-4603-bb45-274d353d0c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:19.985216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-467f067b-7fec-4603-bb45-274d353d0c60 None None] MFA Rules not processed for user `f1f63c450ef64400b86bee86b63bc200`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:19.985216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-467f067b-7fec-4603-bb45-274d353d0c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:20.000671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2dcc28aa1924987b1af652e08971f54', 'request_id': 'req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:20.012869 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] The token cache is being invalidate because project e2dcc28aa1924987b1af652e08971f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:20.019613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:20.019613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:20.019613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1393/2789] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:19 2022] DELETE /identity/v3/projects/e2dcc28aa1924987b1af652e08971f54 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:20.019613 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:20.020710 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:20.024655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:20.048823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1397/2790] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:20.060606 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f067b-7fec-4603-bb45-274d353d0c60 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:20.062428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-467f067b-7fec-4603-bb45-274d353d0c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:20.087998 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028afc1f-b4bd-4921-8c13-f7d0b9ff578e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:20.094153 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:20.095161 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:20.095428 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:20.095701 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] PATH_INFO: `/v3/users/db3eb5f2b0ed4c10a0b3632325b6b675` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:20.097035 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3eb5f2b0ed4c10a0b3632325b6b675)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:20.099462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:20.131161 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f067b-7fec-4603-bb45-274d353d0c60 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:20.135404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d137ae-12c9-4b0c-ba98-51cc87277e7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e95d295337b4472be946d252ac3bd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:20.136018 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d137ae-12c9-4b0c-ba98-51cc87277e7a service glance] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:20.136288 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d137ae-12c9-4b0c-ba98-51cc87277e7a service glance] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:20.136577 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d137ae-12c9-4b0c-ba98-51cc87277e7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:20.145682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d137ae-12c9-4b0c-ba98-51cc87277e7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:20.147769 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d137ae-12c9-4b0c-ba98-51cc87277e7a service glance] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:20.157284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1398/2791] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:20.311985 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3eb5f2b0ed4c10a0b3632325b6b675'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:20.401149 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3eb5f2b0ed4c10a0b3632325b6b675'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:20.413170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3eb5f2b0ed4c10a0b3632325b6b675'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:20.414312 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3eb5f2b0ed4c10a0b3632325b6b675'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:20.415249 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b94159d5-9bd2-485b-af34-4e0d40fa1df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3eb5f2b0ed4c10a0b3632325b6b675'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:20.439326 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1394/2792] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:20 2022] DELETE /identity/v3/users/db3eb5f2b0ed4c10a0b3632325b6b675 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:20.871541 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d137ae-12c9-4b0c-ba98-51cc87277e7a service glance] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:20.883680 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d137ae-12c9-4b0c-ba98-51cc87277e7a service glance] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:20.941128 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:20.941836 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:20.942117 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:20.942380 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] PATH_INFO: `/v3/projects/ba845c07da8e438f8b1d6e877f9c2b02` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:20.943512 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba845c07da8e438f8b1d6e877f9c2b02)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:20.945452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:21.061015 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba845c07da8e438f8b1d6e877f9c2b02', 'request_id': 'req-0fbcce16-77cc-46c2-baf6-d6b845b9022c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:21.066648 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba845c07da8e438f8b1d6e877f9c2b02', 'request_id': 'req-0fbcce16-77cc-46c2-baf6-d6b845b9022c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:21.188268 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba845c07da8e438f8b1d6e877f9c2b02', 'request_id': 'req-0fbcce16-77cc-46c2-baf6-d6b845b9022c'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:21.199996 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] The token cache is being invalidate because project ba845c07da8e438f8b1d6e877f9c2b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:21.200279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:21.200929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-0fbcce16-77cc-46c2-baf6-d6b845b9022c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:21.202382 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1399/2793] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:20 2022] DELETE /identity/v3/projects/ba845c07da8e438f8b1d6e877f9c2b02 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:21.207477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafc297-5600-4ba8-8602-aee3aca2dd4c None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:21.207940 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafc297-5600-4ba8-8602-aee3aca2dd4c None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:21.208037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafc297-5600-4ba8-8602-aee3aca2dd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:21.227823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-fdafc297-5600-4ba8-8602-aee3aca2dd4c None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:21.229039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fdafc297-5600-4ba8-8602-aee3aca2dd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:21.276648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:21.288266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1395/2794] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:21.297287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6afd1-3f88-4bac-a1be-f5eda71b58f5 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:21.297566 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6afd1-3f88-4bac-a1be-f5eda71b58f5 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:21.297896 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6afd1-3f88-4bac-a1be-f5eda71b58f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:21.317655 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-eeb6afd1-3f88-4bac-a1be-f5eda71b58f5 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:21.318841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-eeb6afd1-3f88-4bac-a1be-f5eda71b58f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:21.326855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1400/2795] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:21.330977 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafc297-5600-4ba8-8602-aee3aca2dd4c None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:21.341178 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafc297-5600-4ba8-8602-aee3aca2dd4c None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:21.345699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd20f77-d6b4-440c-aa87-486b723679f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:21.346279 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd20f77-d6b4-440c-aa87-486b723679f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:21.346550 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd20f77-d6b4-440c-aa87-486b723679f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:21.346815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd20f77-d6b4-440c-aa87-486b723679f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:21.347480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd20f77-d6b4-440c-aa87-486b723679f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:17:21.347775 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd20f77-d6b4-440c-aa87-486b723679f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:21.349984 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd20f77-d6b4-440c-aa87-486b723679f1 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:21.361061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1396/2796] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:17:21.365305 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6afd1-3f88-4bac-a1be-f5eda71b58f5 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:21.374401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6afd1-3f88-4bac-a1be-f5eda71b58f5 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:21.379114 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f83fbe-babf-4992-8543-a5f6d3617d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:21.379701 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f83fbe-babf-4992-8543-a5f6d3617d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:21.379973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f83fbe-babf-4992-8543-a5f6d3617d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:21.380240 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f83fbe-babf-4992-8543-a5f6d3617d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:21.380888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f83fbe-babf-4992-8543-a5f6d3617d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:21.382831 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f83fbe-babf-4992-8543-a5f6d3617d65 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:21.388046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:21.461519 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1401/2797] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:21 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:21.465622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd20f77-d6b4-440c-aa87-486b723679f1 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:21.476684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd20f77-d6b4-440c-aa87-486b723679f1 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:21.536210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb17fe5-b085-47e9-ae79-eb0ede3ad8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:21.537507 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb17fe5-b085-47e9-ae79-eb0ede3ad8b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:21.537766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb17fe5-b085-47e9-ae79-eb0ede3ad8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:21.538030 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb17fe5-b085-47e9-ae79-eb0ede3ad8b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:21.538841 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb17fe5-b085-47e9-ae79-eb0ede3ad8b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:21.540622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb17fe5-b085-47e9-ae79-eb0ede3ad8b3 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:21.638236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1397/2798] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:21 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:21.639594 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f83fbe-babf-4992-8543-a5f6d3617d65 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:21.651330 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f83fbe-babf-4992-8543-a5f6d3617d65 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:21.656166 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5f782-7d2d-4e7b-bce6-ff4b269d6726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:21.656804 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f782-7d2d-4e7b-bce6-ff4b269d6726 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:21.657234 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f782-7d2d-4e7b-bce6-ff4b269d6726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:21.657513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f782-7d2d-4e7b-bce6-ff4b269d6726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:21.658170 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5f782-7d2d-4e7b-bce6-ff4b269d6726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:21.659800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5f782-7d2d-4e7b-bce6-ff4b269d6726 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:21.672678 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1402/2799] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:21.676774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb17fe5-b085-47e9-ae79-eb0ede3ad8b3 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:21.687050 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb17fe5-b085-47e9-ae79-eb0ede3ad8b3 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:21.691667 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8bff8-9c3c-4b3c-b544-993ec3e2784c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:21.692527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8bff8-9c3c-4b3c-b544-993ec3e2784c admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:21.692883 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8bff8-9c3c-4b3c-b544-993ec3e2784c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:21.693781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8bff8-9c3c-4b3c-b544-993ec3e2784c admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:21.694487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8bff8-9c3c-4b3c-b544-993ec3e2784c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:21.696142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8bff8-9c3c-4b3c-b544-993ec3e2784c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:21.709353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1398/2800] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:21.713590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5f782-7d2d-4e7b-bce6-ff4b269d6726 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:21.728388 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5f782-7d2d-4e7b-bce6-ff4b269d6726 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:21.733302 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2467f06-dfed-4cee-b4ff-cf5e5313bfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:21.733965 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2467f06-dfed-4cee-b4ff-cf5e5313bfd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:21.734265 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2467f06-dfed-4cee-b4ff-cf5e5313bfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:21.734535 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2467f06-dfed-4cee-b4ff-cf5e5313bfd2 admin admin] PATH_INFO: `/v3/projects/9ffbfe5a6c5b46efba6da7cfa85e5c71/users/c657f66b633144e8bc6d53b20984365f/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:21.750458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2467f06-dfed-4cee-b4ff-cf5e5313bfd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffbfe5a6c5b46efba6da7cfa85e5c71, user_id=c657f66b633144e8bc6d53b20984365f, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:21.753391 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2467f06-dfed-4cee-b4ff-cf5e5313bfd2 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:21.846373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1403/2801] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:21 2022] PUT /identity/v3/projects/9ffbfe5a6c5b46efba6da7cfa85e5c71/users/c657f66b633144e8bc6d53b20984365f/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:21.857017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d93a8e-5ac3-4864-bdf8-385ed2a14b64 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:21.857480 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d93a8e-5ac3-4864-bdf8-385ed2a14b64 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:21.857748 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d93a8e-5ac3-4864-bdf8-385ed2a14b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:21.887142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-e8d93a8e-5ac3-4864-bdf8-385ed2a14b64 None None] MFA Rules not processed for user `c657f66b633144e8bc6d53b20984365f`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:21.888325 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e8d93a8e-5ac3-4864-bdf8-385ed2a14b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:21.949779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1399/2802] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:21.954369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2467f06-dfed-4cee-b4ff-cf5e5313bfd2 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:22.022275 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2467f06-dfed-4cee-b4ff-cf5e5313bfd2 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:22.027329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540db2a-7ffa-4190-a1ba-dffb3d19c2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:22.027863 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540db2a-7ffa-4190-a1ba-dffb3d19c2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:22.028280 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540db2a-7ffa-4190-a1ba-dffb3d19c2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:22.028590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540db2a-7ffa-4190-a1ba-dffb3d19c2b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:22.029238 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540db2a-7ffa-4190-a1ba-dffb3d19c2b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:22.031270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540db2a-7ffa-4190-a1ba-dffb3d19c2b3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:22.151627 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1404/2803] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:21 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:22.155944 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d93a8e-5ac3-4864-bdf8-385ed2a14b64 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:22.167317 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d93a8e-5ac3-4864-bdf8-385ed2a14b64 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:22.213847 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:22.225921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd551c-2549-4d06-9862-aeea1897aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:22.225921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd551c-2549-4d06-9862-aeea1897aa24 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:22.225921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd551c-2549-4d06-9862-aeea1897aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:22.225921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd551c-2549-4d06-9862-aeea1897aa24 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:22.226485 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd551c-2549-4d06-9862-aeea1897aa24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:22.227807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd551c-2549-4d06-9862-aeea1897aa24 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:22.326541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1400/2804] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:22 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:22.330135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540db2a-7ffa-4190-a1ba-dffb3d19c2b3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:22.342012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540db2a-7ffa-4190-a1ba-dffb3d19c2b3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:22.348315 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe88e8b-2134-457c-90e3-e28af7b897e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:22.348980 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe88e8b-2134-457c-90e3-e28af7b897e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:22.349259 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe88e8b-2134-457c-90e3-e28af7b897e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:22.350195 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe88e8b-2134-457c-90e3-e28af7b897e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:22.350839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe88e8b-2134-457c-90e3-e28af7b897e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:22.352805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe88e8b-2134-457c-90e3-e28af7b897e1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:22.365846 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1405/2805] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:22.369392 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd551c-2549-4d06-9862-aeea1897aa24 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:22.379455 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd551c-2549-4d06-9862-aeea1897aa24 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:22.384684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a5514-66c3-4f27-8579-48bf750b2a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:22.385266 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5514-66c3-4f27-8579-48bf750b2a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:22.385540 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5514-66c3-4f27-8579-48bf750b2a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:22.385807 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5514-66c3-4f27-8579-48bf750b2a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:22.386437 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a5514-66c3-4f27-8579-48bf750b2a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:22.388068 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a5514-66c3-4f27-8579-48bf750b2a3d admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:22.398547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1401/2806] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:22.402987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe88e8b-2134-457c-90e3-e28af7b897e1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:22.415439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe88e8b-2134-457c-90e3-e28af7b897e1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:22.420793 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10272bf-794f-4d2a-a7ad-4f9e7f86331c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:22.421461 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10272bf-794f-4d2a-a7ad-4f9e7f86331c admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:22.421740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10272bf-794f-4d2a-a7ad-4f9e7f86331c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:22.422012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10272bf-794f-4d2a-a7ad-4f9e7f86331c admin admin] PATH_INFO: `/v3/projects/ccab314f66df4faf87d9171a83ec6a4b/users/9d0fd405a3d041999dd7bb05f49c7a26/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:22.438774 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10272bf-794f-4d2a-a7ad-4f9e7f86331c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccab314f66df4faf87d9171a83ec6a4b, user_id=9d0fd405a3d041999dd7bb05f49c7a26, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:22.441560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10272bf-794f-4d2a-a7ad-4f9e7f86331c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:22.511085 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1406/2807] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:22 2022] PUT /identity/v3/projects/ccab314f66df4faf87d9171a83ec6a4b/users/9d0fd405a3d041999dd7bb05f49c7a26/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:22.523760 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dd897-0022-470e-b7bb-b5d16a433c68 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:22.524039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dd897-0022-470e-b7bb-b5d16a433c68 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:22.524335 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dd897-0022-470e-b7bb-b5d16a433c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:22.553593 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2a8dd897-0022-470e-b7bb-b5d16a433c68 None None] MFA Rules not processed for user `9d0fd405a3d041999dd7bb05f49c7a26`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:22.554809 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2a8dd897-0022-470e-b7bb-b5d16a433c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:22.619002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1402/2808] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:22.623455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10272bf-794f-4d2a-a7ad-4f9e7f86331c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:22.689278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10272bf-794f-4d2a-a7ad-4f9e7f86331c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:22.694654 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6765b-eaf9-4967-b1fe-386a2198bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:22.695255 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6765b-eaf9-4967-b1fe-386a2198bc70 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:22.695531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6765b-eaf9-4967-b1fe-386a2198bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:22.695800 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6765b-eaf9-4967-b1fe-386a2198bc70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:22.696428 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6765b-eaf9-4967-b1fe-386a2198bc70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:22.698405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6765b-eaf9-4967-b1fe-386a2198bc70 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:22.703822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:22.775759 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1407/2809] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:22 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:17:22.780638 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dd897-0022-470e-b7bb-b5d16a433c68 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:22.804780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dd897-0022-470e-b7bb-b5d16a433c68 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:22.867686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b655e1-7450-46dc-a50c-d9e08d603a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:22.868298 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b655e1-7450-46dc-a50c-d9e08d603a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:22.868603 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b655e1-7450-46dc-a50c-d9e08d603a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:22.868876 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b655e1-7450-46dc-a50c-d9e08d603a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:22.869566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b655e1-7450-46dc-a50c-d9e08d603a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:22.871412 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b655e1-7450-46dc-a50c-d9e08d603a31 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:22.978463 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1403/2810] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:22 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:22.982359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6765b-eaf9-4967-b1fe-386a2198bc70 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:22.993064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6765b-eaf9-4967-b1fe-386a2198bc70 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:22.997536 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b5007-7d7f-47f8-9f50-3dd174ea278e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:22.998118 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b5007-7d7f-47f8-9f50-3dd174ea278e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:22.998393 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b5007-7d7f-47f8-9f50-3dd174ea278e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:22.998659 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b5007-7d7f-47f8-9f50-3dd174ea278e admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:22.999287 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b5007-7d7f-47f8-9f50-3dd174ea278e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:23.000930 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b5007-7d7f-47f8-9f50-3dd174ea278e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:23.011653 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1408/2811] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:23.015828 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b655e1-7450-46dc-a50c-d9e08d603a31 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:23.025863 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b655e1-7450-46dc-a50c-d9e08d603a31 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:23.030460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2b1b-82ea-4446-9ab8-416df620de51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:23.031092 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2b1b-82ea-4446-9ab8-416df620de51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:23.031367 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2b1b-82ea-4446-9ab8-416df620de51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:23.031635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2b1b-82ea-4446-9ab8-416df620de51 admin admin] PATH_INFO: `/v3/projects/cb539bc37e8f48b2b518b5e8b82f9281/users/282205b9f5544dc3b50b130d7347749c/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:23.048541 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2b1b-82ea-4446-9ab8-416df620de51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb539bc37e8f48b2b518b5e8b82f9281, user_id=282205b9f5544dc3b50b130d7347749c, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:23.051850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2b1b-82ea-4446-9ab8-416df620de51 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:23.121854 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1404/2812] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:23 2022] PUT /identity/v3/projects/cb539bc37e8f48b2b518b5e8b82f9281/users/282205b9f5544dc3b50b130d7347749c/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:23.125730 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b5007-7d7f-47f8-9f50-3dd174ea278e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:23.136116 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b5007-7d7f-47f8-9f50-3dd174ea278e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:23.191053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e84b91-df86-49c6-b6bb-bb4790b3eea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:23.191639 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e84b91-df86-49c6-b6bb-bb4790b3eea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:23.191910 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e84b91-df86-49c6-b6bb-bb4790b3eea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:23.192277 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e84b91-df86-49c6-b6bb-bb4790b3eea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:23.192935 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e84b91-df86-49c6-b6bb-bb4790b3eea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:23.194580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e84b91-df86-49c6-b6bb-bb4790b3eea8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:23.204042 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1409/2813] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:17:23.208581 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2b1b-82ea-4446-9ab8-416df620de51 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:23.218912 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2b1b-82ea-4446-9ab8-416df620de51 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:23.223484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e496d8-e994-4312-8167-76f773e892ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:23.224109 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e496d8-e994-4312-8167-76f773e892ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:23.224900 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e496d8-e994-4312-8167-76f773e892ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:23.225163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e496d8-e994-4312-8167-76f773e892ae admin admin] PATH_INFO: `/v3/projects/cb539bc37e8f48b2b518b5e8b82f9281/users/282205b9f5544dc3b50b130d7347749c/roles/af39f6211a7244df8d429c0e561b9b2f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:23.227021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e496d8-e994-4312-8167-76f773e892ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb539bc37e8f48b2b518b5e8b82f9281, user_id=282205b9f5544dc3b50b130d7347749c, role_id=af39f6211a7244df8d429c0e561b9b2f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:23.229781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e496d8-e994-4312-8167-76f773e892ae admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:23.239677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1405/2814] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:23 2022] PUT /identity/v3/projects/cb539bc37e8f48b2b518b5e8b82f9281/users/282205b9f5544dc3b50b130d7347749c/roles/af39f6211a7244df8d429c0e561b9b2f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:23.251145 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986a9a2-6249-4067-a17c-2b0e8483cef8 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:23.251431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986a9a2-6249-4067-a17c-2b0e8483cef8 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:23.251703 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986a9a2-6249-4067-a17c-2b0e8483cef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:23.280834 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7986a9a2-6249-4067-a17c-2b0e8483cef8 None None] MFA Rules not processed for user `282205b9f5544dc3b50b130d7347749c`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:23.282029 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7986a9a2-6249-4067-a17c-2b0e8483cef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:23.349135 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1410/2815] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:23.361740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e496d8-e994-4312-8167-76f773e892ae admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:23.363592 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-29e496d8-e994-4312-8167-76f773e892ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:23.384045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:23.447477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e496d8-e994-4312-8167-76f773e892ae admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:23.451163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c165ca-7507-4e23-90f8-cdd47179a03e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:23.453105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c165ca-7507-4e23-90f8-cdd47179a03e service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:23.453105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c165ca-7507-4e23-90f8-cdd47179a03e service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:23.453105 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c165ca-7507-4e23-90f8-cdd47179a03e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:23.507684 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c165ca-7507-4e23-90f8-cdd47179a03e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:23.509143 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c165ca-7507-4e23-90f8-cdd47179a03e service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:23.518071 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1406/2816] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:25.445778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986a9a2-6249-4067-a17c-2b0e8483cef8 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:25.455306 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986a9a2-6249-4067-a17c-2b0e8483cef8 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:25.459167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4b230-aa45-493b-acec-8a9dd5816926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:25.459749 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4b230-aa45-493b-acec-8a9dd5816926 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:25.460020 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4b230-aa45-493b-acec-8a9dd5816926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:25.460282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4b230-aa45-493b-acec-8a9dd5816926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:25.513718 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:25.516254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4b230-aa45-493b-acec-8a9dd5816926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:25.517747 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4b230-aa45-493b-acec-8a9dd5816926 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:25.526635 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1411/2817] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:26.842114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c165ca-7507-4e23-90f8-cdd47179a03e service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:26.850979 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c165ca-7507-4e23-90f8-cdd47179a03e service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:26.854774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed9a80-f9d4-44cd-a310-79f4f4372d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:26.855371 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed9a80-f9d4-44cd-a310-79f4f4372d04 service cinder] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:26.855648 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed9a80-f9d4-44cd-a310-79f4f4372d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:26.855931 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed9a80-f9d4-44cd-a310-79f4f4372d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:26.866454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed9a80-f9d4-44cd-a310-79f4f4372d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:26.867927 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed9a80-f9d4-44cd-a310-79f4f4372d04 service cinder] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:26.878439 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1407/2818] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:30.288251 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b027d-3ded-4d6b-aeb2-05ef778b0706 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:30.288770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b027d-3ded-4d6b-aeb2-05ef778b0706 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:30.288869 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b027d-3ded-4d6b-aeb2-05ef778b0706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:30.311843 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6e3b027d-3ded-4d6b-aeb2-05ef778b0706 None None] MFA Rules not processed for user `4d85e61124af4c69919f33e647e14c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:30.313207 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6e3b027d-3ded-4d6b-aeb2-05ef778b0706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:30.370727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1412/2819] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:31.423574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed9a80-f9d4-44cd-a310-79f4f4372d04 service cinder] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:31.485203 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed9a80-f9d4-44cd-a310-79f4f4372d04 service cinder] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:31.489656 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:31.490269 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:31.491173 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:31.491440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] PATH_INFO: `/v3/users/c657f66b633144e8bc6d53b20984365f` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:31.492562 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c657f66b633144e8bc6d53b20984365f)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:31.494440 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:31.662267 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:31.670041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c657f66b633144e8bc6d53b20984365f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:31.771273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c657f66b633144e8bc6d53b20984365f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:31.779361 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c657f66b633144e8bc6d53b20984365f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:31.780025 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c657f66b633144e8bc6d53b20984365f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:31.780686 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c657f66b633144e8bc6d53b20984365f'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:31.799740 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1408/2820] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:31 2022] DELETE /identity/v3/users/c657f66b633144e8bc6d53b20984365f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:31.804218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3a1e6-c096-405a-95f4-11d97fedf1c9 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:31.804717 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3a1e6-c096-405a-95f4-11d97fedf1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:31.804823 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3a1e6-c096-405a-95f4-11d97fedf1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:31.823888 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-fef3a1e6-c096-405a-95f4-11d97fedf1c9 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:31.825291 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fef3a1e6-c096-405a-95f4-11d97fedf1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:31.888916 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1413/2821] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:31.901853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:31.903268 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:31.967388 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9488d-4338-4da2-98b0-ce5ed6201c47 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:31.971129 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73609ba3-8df2-4b83-9dd1-c0f78e40775d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:31.971724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73609ba3-8df2-4b83-9dd1-c0f78e40775d service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:31.972001 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73609ba3-8df2-4b83-9dd1-c0f78e40775d service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:31.972270 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73609ba3-8df2-4b83-9dd1-c0f78e40775d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:31.981946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73609ba3-8df2-4b83-9dd1-c0f78e40775d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:31.983368 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73609ba3-8df2-4b83-9dd1-c0f78e40775d service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:31.993041 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1409/2822] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:32.421396 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3a1e6-c096-405a-95f4-11d97fedf1c9 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:32.425509 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-fef3a1e6-c096-405a-95f4-11d97fedf1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:32.436797 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3a1e6-c096-405a-95f4-11d97fedf1c9 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:32.442608 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:32.443605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:32.443939 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:32.444271 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] PATH_INFO: `/v3/projects/9ffbfe5a6c5b46efba6da7cfa85e5c71` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:32.445720 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ffbfe5a6c5b46efba6da7cfa85e5c71)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:32.447740 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:32.548096 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ffbfe5a6c5b46efba6da7cfa85e5c71', 'request_id': 'req-ccb488ab-f575-4841-8b6d-040e717419bf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:32.553507 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ffbfe5a6c5b46efba6da7cfa85e5c71', 'request_id': 'req-ccb488ab-f575-4841-8b6d-040e717419bf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:32.605115 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ffbfe5a6c5b46efba6da7cfa85e5c71', 'request_id': 'req-ccb488ab-f575-4841-8b6d-040e717419bf'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:32.616595 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] The token cache is being invalidate because project 9ffbfe5a6c5b46efba6da7cfa85e5c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:32.617355 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:32.617529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:32.619008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1414/2823] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:32 2022] DELETE /identity/v3/projects/9ffbfe5a6c5b46efba6da7cfa85e5c71 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:32.623016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73609ba3-8df2-4b83-9dd1-c0f78e40775d service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:32.632817 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73609ba3-8df2-4b83-9dd1-c0f78e40775d service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:32.688303 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:32.688961 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:32.689334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:32.689609 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] PATH_INFO: `/v3/users/9d0fd405a3d041999dd7bb05f49c7a26` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:32.690724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d0fd405a3d041999dd7bb05f49c7a26)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:32.692909 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:32.905635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d0fd405a3d041999dd7bb05f49c7a26'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:32.967438 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d0fd405a3d041999dd7bb05f49c7a26'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:32.971209 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:32.987286 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d0fd405a3d041999dd7bb05f49c7a26'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:32.988499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d0fd405a3d041999dd7bb05f49c7a26'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:32.989265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d0fd405a3d041999dd7bb05f49c7a26'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:33.010300 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1410/2824] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:32 2022] DELETE /identity/v3/users/9d0fd405a3d041999dd7bb05f49c7a26 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:33.638325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:33.639813 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:33.653282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb488ab-f575-4841-8b6d-040e717419bf admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:33.674301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:33.722840 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:33.723485 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:33.723761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:33.724028 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] PATH_INFO: `/v3/projects/ccab314f66df4faf87d9171a83ec6a4b` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:33.725918 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccab314f66df4faf87d9171a83ec6a4b)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:33.728158 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:33.837243 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccab314f66df4faf87d9171a83ec6a4b', 'request_id': 'req-b8bb22b2-c417-4eb0-bb05-81f0230085b9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:33.842833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccab314f66df4faf87d9171a83ec6a4b', 'request_id': 'req-b8bb22b2-c417-4eb0-bb05-81f0230085b9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:33.886632 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccab314f66df4faf87d9171a83ec6a4b', 'request_id': 'req-b8bb22b2-c417-4eb0-bb05-81f0230085b9'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:33.901613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] The token cache is being invalidate because project ccab314f66df4faf87d9171a83ec6a4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:33.901911 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:33.902778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:33.905886 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1415/2825] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:33 2022] DELETE /identity/v3/projects/ccab314f66df4faf87d9171a83ec6a4b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:33.909762 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:33.911236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:33.934160 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bc60b-d7cf-44c5-94b4-b7ad85b28e2c admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:33.999450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:34.000578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:34.000578 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:34.000829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] PATH_INFO: `/v3/users/282205b9f5544dc3b50b130d7347749c` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:34.001853 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=282205b9f5544dc3b50b130d7347749c)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:34.003754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:34.223963 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282205b9f5544dc3b50b130d7347749c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:34.308602 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282205b9f5544dc3b50b130d7347749c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:34.318021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282205b9f5544dc3b50b130d7347749c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:34.318170 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282205b9f5544dc3b50b130d7347749c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:34.320080 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-86171e89-18eb-463b-8423-7cee51c83edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282205b9f5544dc3b50b130d7347749c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:34.343622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1411/2826] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:33 2022] DELETE /identity/v3/users/282205b9f5544dc3b50b130d7347749c => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:17:34.820030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:34.822138 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:34.857808 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb22b2-c417-4eb0-bb05-81f0230085b9 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:34.932301 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:34.932957 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:34.933236 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:34.933505 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] PATH_INFO: `/v3/projects/cb539bc37e8f48b2b518b5e8b82f9281` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:34.934625 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb539bc37e8f48b2b518b5e8b82f9281)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:34.936572 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:35.039839 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb539bc37e8f48b2b518b5e8b82f9281', 'request_id': 'req-d60e7525-4aca-4ec2-8917-604f15c78e05'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:35.045347 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:35.046867 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb539bc37e8f48b2b518b5e8b82f9281', 'request_id': 'req-d60e7525-4aca-4ec2-8917-604f15c78e05'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:35.132487 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb539bc37e8f48b2b518b5e8b82f9281', 'request_id': 'req-d60e7525-4aca-4ec2-8917-604f15c78e05'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:35.146030 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] The token cache is being invalidate because project cb539bc37e8f48b2b518b5e8b82f9281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:35.146294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:35.147057 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60e7525-4aca-4ec2-8917-604f15c78e05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:35.148801 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1416/2827] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:34 2022] DELETE /identity/v3/projects/cb539bc37e8f48b2b518b5e8b82f9281 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:35.153719 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e47ac-ba8d-4ffd-9a57-125de5ad31bf None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.154264 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e47ac-ba8d-4ffd-9a57-125de5ad31bf None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.154390 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e47ac-ba8d-4ffd-9a57-125de5ad31bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.175166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-8a4e47ac-ba8d-4ffd-9a57-125de5ad31bf None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:35.176376 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-8a4e47ac-ba8d-4ffd-9a57-125de5ad31bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:35.233252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:35.247365 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1412/2828] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:35.255955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e06226-c7b0-4717-b629-781d9c4eaa9e None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.256239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e06226-c7b0-4717-b629-781d9c4eaa9e None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.256529 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e06226-c7b0-4717-b629-781d9c4eaa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.277405 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-c3e06226-c7b0-4717-b629-781d9c4eaa9e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:35.283257 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c3e06226-c7b0-4717-b629-781d9c4eaa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:35.286955 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1417/2829] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:35.291831 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e47ac-ba8d-4ffd-9a57-125de5ad31bf None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:35.308007 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e47ac-ba8d-4ffd-9a57-125de5ad31bf None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:35.313259 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a2ee8-4781-4f54-ade9-3c593544a59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:35.313856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a2ee8-4781-4f54-ade9-3c593544a59b admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.314133 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a2ee8-4781-4f54-ade9-3c593544a59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.314399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a2ee8-4781-4f54-ade9-3c593544a59b admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.317483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a2ee8-4781-4f54-ade9-3c593544a59b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:17:35.317483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a2ee8-4781-4f54-ade9-3c593544a59b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:35.317483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a2ee8-4781-4f54-ade9-3c593544a59b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:35.328713 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1413/2830] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:17:35.333261 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e06226-c7b0-4717-b629-781d9c4eaa9e None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:35.342682 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e06226-c7b0-4717-b629-781d9c4eaa9e None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:35.347314 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f067ec-816f-4a3f-bdd8-a37e2d475c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:35.347902 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f067ec-816f-4a3f-bdd8-a37e2d475c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.348169 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f067ec-816f-4a3f-bdd8-a37e2d475c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.348431 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f067ec-816f-4a3f-bdd8-a37e2d475c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.349087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f067ec-816f-4a3f-bdd8-a37e2d475c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:35.351829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f067ec-816f-4a3f-bdd8-a37e2d475c28 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:35.459926 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1418/2831] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:17:35 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:17:35.464363 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a2ee8-4781-4f54-ade9-3c593544a59b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:35.474357 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a2ee8-4781-4f54-ade9-3c593544a59b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:35.533600 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d01d45-245d-4770-8ce2-75485e7f8fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:35.534211 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d01d45-245d-4770-8ce2-75485e7f8fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.534483 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d01d45-245d-4770-8ce2-75485e7f8fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.535974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d01d45-245d-4770-8ce2-75485e7f8fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.535974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d01d45-245d-4770-8ce2-75485e7f8fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:35.537265 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d01d45-245d-4770-8ce2-75485e7f8fe2 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:35.612850 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1414/2832] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:17:35 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:17:35.617776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f067ec-816f-4a3f-bdd8-a37e2d475c28 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:35.628531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f067ec-816f-4a3f-bdd8-a37e2d475c28 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:35.634414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d79fd-d368-4eb5-b6a1-bc4380aaa45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:35.635129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d79fd-d368-4eb5-b6a1-bc4380aaa45e admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.635513 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d79fd-d368-4eb5-b6a1-bc4380aaa45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.635786 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d79fd-d368-4eb5-b6a1-bc4380aaa45e admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.637023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d79fd-d368-4eb5-b6a1-bc4380aaa45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:35.638054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d79fd-d368-4eb5-b6a1-bc4380aaa45e admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:35.649123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1419/2833] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:35.653230 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d01d45-245d-4770-8ce2-75485e7f8fe2 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:35.665448 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d01d45-245d-4770-8ce2-75485e7f8fe2 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:35.670166 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04f07f-ac57-4b0b-a113-5fc471e973a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:35.670791 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04f07f-ac57-4b0b-a113-5fc471e973a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.671066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04f07f-ac57-4b0b-a113-5fc471e973a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.671334 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04f07f-ac57-4b0b-a113-5fc471e973a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.671975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04f07f-ac57-4b0b-a113-5fc471e973a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:35.673603 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04f07f-ac57-4b0b-a113-5fc471e973a7 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:35.685380 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1415/2834] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:17:35.689531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d79fd-d368-4eb5-b6a1-bc4380aaa45e admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:35.700818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d79fd-d368-4eb5-b6a1-bc4380aaa45e admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:35.705725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd91d0-ce84-437b-9212-79200c8f1196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:35.707340 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd91d0-ce84-437b-9212-79200c8f1196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.707649 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd91d0-ce84-437b-9212-79200c8f1196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.707919 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd91d0-ce84-437b-9212-79200c8f1196 admin admin] PATH_INFO: `/v3/projects/a7a885b9c1574a8eb0a62b5253c9f8c2/users/6ae6453ca9ec4a4eb3ec2e25900810cc/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.726299 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd91d0-ce84-437b-9212-79200c8f1196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a885b9c1574a8eb0a62b5253c9f8c2, user_id=6ae6453ca9ec4a4eb3ec2e25900810cc, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:35.727948 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd91d0-ce84-437b-9212-79200c8f1196 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:35.801517 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1420/2835] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:17:35 2022] PUT /identity/v3/projects/a7a885b9c1574a8eb0a62b5253c9f8c2/users/6ae6453ca9ec4a4eb3ec2e25900810cc/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:35.807402 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee06ab-ee30-4ddc-a0d0-451e24089a90 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.807683 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee06ab-ee30-4ddc-a0d0-451e24089a90 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.807951 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee06ab-ee30-4ddc-a0d0-451e24089a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.826989 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-91ee06ab-ee30-4ddc-a0d0-451e24089a90 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:35.828179 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-91ee06ab-ee30-4ddc-a0d0-451e24089a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:35.886442 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1416/2836] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:35.896031 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd91d0-ce84-437b-9212-79200c8f1196 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:35.899932 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-31bd91d0-ce84-437b-9212-79200c8f1196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:35.956230 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd91d0-ce84-437b-9212-79200c8f1196 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:35.960341 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5edc1-94db-4c5b-b94b-3bde3868d1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:35.960983 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5edc1-94db-4c5b-b94b-3bde3868d1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:35.961258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5edc1-94db-4c5b-b94b-3bde3868d1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:35.961523 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5edc1-94db-4c5b-b94b-3bde3868d1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:35.970445 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5edc1-94db-4c5b-b94b-3bde3868d1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:35.971871 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5edc1-94db-4c5b-b94b-3bde3868d1f1 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:35.981667 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1421/2837] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:37.065915 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658229b-b96b-4229-a027-291eae98e1de None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:37.066476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658229b-b96b-4229-a027-291eae98e1de None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:37.066476 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658229b-b96b-4229-a027-291eae98e1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:37.095443 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-f658229b-b96b-4229-a027-291eae98e1de None None] MFA Rules not processed for user `6ae6453ca9ec4a4eb3ec2e25900810cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:37.096691 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-f658229b-b96b-4229-a027-291eae98e1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:37.151238 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:37.169698 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1417/2838] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:37.181154 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5edc1-94db-4c5b-b94b-3bde3868d1f1 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:37.182773 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3aa5edc1-94db-4c5b-b94b-3bde3868d1f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:37.243185 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5edc1-94db-4c5b-b94b-3bde3868d1f1 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:37.246810 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b470bc-e71f-4037-9823-ec712e30faa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:37.247404 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b470bc-e71f-4037-9823-ec712e30faa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:37.247698 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b470bc-e71f-4037-9823-ec712e30faa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:37.247956 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b470bc-e71f-4037-9823-ec712e30faa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:37.256873 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b470bc-e71f-4037-9823-ec712e30faa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:37.258232 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b470bc-e71f-4037-9823-ec712e30faa0 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:37.266814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1422/2839] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:38.899168 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df011ca2-8cb7-49f2-8c3d-5297dd329231 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:38.899690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df011ca2-8cb7-49f2-8c3d-5297dd329231 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:38.899690 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df011ca2-8cb7-49f2-8c3d-5297dd329231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:38.918243 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-df011ca2-8cb7-49f2-8c3d-5297dd329231 None None] MFA Rules not processed for user `6ae6453ca9ec4a4eb3ec2e25900810cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:38.919427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-df011ca2-8cb7-49f2-8c3d-5297dd329231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:38.926139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1418/2840] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:47.262959 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:48.912747 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=8 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:17:53.013534 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b470bc-e71f-4037-9823-ec712e30faa0 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:53.015090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-45b470bc-e71f-4037-9823-ec712e30faa0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:53.029448 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b470bc-e71f-4037-9823-ec712e30faa0 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:53.034273 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:53.034877 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:53.035146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:53.035408 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] PATH_INFO: `/v3/users/f1f63c450ef64400b86bee86b63bc200` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:53.041488 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f63c450ef64400b86bee86b63bc200)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:53.041488 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:53.247248 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f63c450ef64400b86bee86b63bc200'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:53.313354 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f63c450ef64400b86bee86b63bc200'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:53.321144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f63c450ef64400b86bee86b63bc200'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:53.321782 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f63c450ef64400b86bee86b63bc200'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:53.322385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f63c450ef64400b86bee86b63bc200'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:53.340278 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1423/2841] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:17:53 2022] DELETE /identity/v3/users/f1f63c450ef64400b86bee86b63bc200 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:53.344866 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262be79-4483-48ef-b8be-8e77cf29f3b6 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:53.345346 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262be79-4483-48ef-b8be-8e77cf29f3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:53.345454 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262be79-4483-48ef-b8be-8e77cf29f3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:53.364309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-b262be79-4483-48ef-b8be-8e77cf29f3b6 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:53.365534 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b262be79-4483-48ef-b8be-8e77cf29f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:53.422672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1419/2842] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:17:53.432218 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:53.433637 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:53.490384 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43264b21-b372-4b40-bffb-f535778cebb1 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:53.494444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eeefea-30ba-440d-90c8-3b608c918efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:53.495025 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeefea-30ba-440d-90c8-3b608c918efc service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:53.495296 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeefea-30ba-440d-90c8-3b608c918efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:53.495559 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeefea-30ba-440d-90c8-3b608c918efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:53.504147 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eeefea-30ba-440d-90c8-3b608c918efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:53.505590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eeefea-30ba-440d-90c8-3b608c918efc service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:53.515342 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1424/2843] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:17:53.899353 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262be79-4483-48ef-b8be-8e77cf29f3b6 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:53.900782 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-b262be79-4483-48ef-b8be-8e77cf29f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:53.919146 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262be79-4483-48ef-b8be-8e77cf29f3b6 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:53.924763 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:53.925431 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:53.925641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:53.925903 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] PATH_INFO: `/v3/projects/8727724683894b19af25c3889e4dad00` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:53.927002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=8727724683894b19af25c3889e4dad00)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:53.928921 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:54.036066 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8727724683894b19af25c3889e4dad00', 'request_id': 'req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:54.042473 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8727724683894b19af25c3889e4dad00', 'request_id': 'req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:54.115139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8727724683894b19af25c3889e4dad00', 'request_id': 'req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:54.127708 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] The token cache is being invalidate because project 8727724683894b19af25c3889e4dad00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:17:54.128002 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:54.128694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:17:54.130236 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1420/2844] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:17:53 2022] DELETE /identity/v3/projects/8727724683894b19af25c3889e4dad00 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:17:56.824078 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4fe90-6a76-4423-a819-ec9b34181e63 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:56.824619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4fe90-6a76-4423-a819-ec9b34181e63 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:56.824727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4fe90-6a76-4423-a819-ec9b34181e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:56.854540 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-d6a4fe90-6a76-4423-a819-ec9b34181e63 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:17:56.855722 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d6a4fe90-6a76-4423-a819-ec9b34181e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:56.918233 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1425/2845] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 18:17:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:17:57.040633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:17:57.042083 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:17:57.111735 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d769ed0-6760-4318-b2c2-ec45d3faf1ed admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:17:57.117745 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc91b69-13b5-4e8c-9d0e-07c4af41d76e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:17:57.118345 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc91b69-13b5-4e8c-9d0e-07c4af41d76e service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:17:57.118623 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc91b69-13b5-4e8c-9d0e-07c4af41d76e service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:17:57.118895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc91b69-13b5-4e8c-9d0e-07c4af41d76e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:17:57.127282 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc91b69-13b5-4e8c-9d0e-07c4af41d76e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:17:57.128930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc91b69-13b5-4e8c-9d0e-07c4af41d76e service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:17:57.137186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1421/2846] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:17:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:18:06.911784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:18:07.133919 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:18:19.408327 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f3587-3b3b-45de-aaed-f505fdd62f13 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:19.408822 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f3587-3b3b-45de-aaed-f505fdd62f13 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:19.408929 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f3587-3b3b-45de-aaed-f505fdd62f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:19.428220 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-6d7f3587-3b3b-45de-aaed-f505fdd62f13 None None] MFA Rules not processed for user `4f0ca85ea7974962b8115428563aef8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:18:19.430046 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6d7f3587-3b3b-45de-aaed-f505fdd62f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:18:19.437163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1426/2847] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 18:18:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:18:19.449210 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc91b69-13b5-4e8c-9d0e-07c4af41d76e service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:19.460103 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc91b69-13b5-4e8c-9d0e-07c4af41d76e service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:19.465199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdf1fb-398f-4f7b-b70d-9010c3d5003a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3375c3134ebe473cb5da29f083321cdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:19.465777 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdf1fb-398f-4f7b-b70d-9010c3d5003a service swift] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:19.466052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdf1fb-398f-4f7b-b70d-9010c3d5003a service swift] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:19.466321 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdf1fb-398f-4f7b-b70d-9010c3d5003a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:19.474399 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdf1fb-398f-4f7b-b70d-9010c3d5003a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:19.476552 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdf1fb-398f-4f7b-b70d-9010c3d5003a service swift] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:19.485517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1422/2848] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 18:18:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 18:18:22.333778 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f3587-3b3b-45de-aaed-f505fdd62f13 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:22.335223 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6d7f3587-3b3b-45de-aaed-f505fdd62f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:18:22.401420 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f3587-3b3b-45de-aaed-f505fdd62f13 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:22.405973 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:22.407272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:22.407272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:22.407272 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] PATH_INFO: `/v3/users/4d85e61124af4c69919f33e647e14c75` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:22.408279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d85e61124af4c69919f33e647e14c75)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:22.410211 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:22.562643 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d85e61124af4c69919f33e647e14c75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:22.600422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d85e61124af4c69919f33e647e14c75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:22.607579 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d85e61124af4c69919f33e647e14c75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:22.608254 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d85e61124af4c69919f33e647e14c75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:22.608945 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-b9e74064-f5ab-4e57-9dca-f992f3aa6280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d85e61124af4c69919f33e647e14c75'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:22.627242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1427/2849] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:18:22 2022] DELETE /identity/v3/users/4d85e61124af4c69919f33e647e14c75 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:18:22.899295 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdf1fb-398f-4f7b-b70d-9010c3d5003a service swift] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:22.911046 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdf1fb-398f-4f7b-b70d-9010c3d5003a service swift] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:22.967316 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:22.967930 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:22.968199 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:22.968462 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] PATH_INFO: `/v3/projects/9e956c74d0ec4521b0a792e0e242b695` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:22.969573 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e956c74d0ec4521b0a792e0e242b695)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:22.971444 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:23.075128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e956c74d0ec4521b0a792e0e242b695', 'request_id': 'req-b2df07e3-be75-4b34-8ffe-daf120dc359c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:23.081104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e956c74d0ec4521b0a792e0e242b695', 'request_id': 'req-b2df07e3-be75-4b34-8ffe-daf120dc359c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:23.160779 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e956c74d0ec4521b0a792e0e242b695', 'request_id': 'req-b2df07e3-be75-4b34-8ffe-daf120dc359c'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:23.172384 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] The token cache is being invalidate because project 9e956c74d0ec4521b0a792e0e242b695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:18:23.172700 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:23.173347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-b2df07e3-be75-4b34-8ffe-daf120dc359c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:18:23.174849 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1423/2850] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:18:22 2022] DELETE /identity/v3/projects/9e956c74d0ec4521b0a792e0e242b695 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:18:23.179881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe4250-3a49-4967-8d16-a430126fb477 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.180190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe4250-3a49-4967-8d16-a430126fb477 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.180452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe4250-3a49-4967-8d16-a430126fb477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.201628 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-68fe4250-3a49-4967-8d16-a430126fb477 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:18:23.202833 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-68fe4250-3a49-4967-8d16-a430126fb477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:18:23.260742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1428/2851] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:18:23.269309 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68d171-cc0f-4993-8ea5-dbc8f574d1b1 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.269596 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68d171-cc0f-4993-8ea5-dbc8f574d1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.269901 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68d171-cc0f-4993-8ea5-dbc8f574d1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.289789 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-ff68d171-cc0f-4993-8ea5-dbc8f574d1b1 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:18:23.290990 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-ff68d171-cc0f-4993-8ea5-dbc8f574d1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:18:23.298812 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1424/2852] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:18:23.302855 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe4250-3a49-4967-8d16-a430126fb477 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:23.312560 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe4250-3a49-4967-8d16-a430126fb477 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:23.317190 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049bf01-1b86-4bca-b494-a4b41562ca07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:23.317770 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049bf01-1b86-4bca-b494-a4b41562ca07 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.318047 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049bf01-1b86-4bca-b494-a4b41562ca07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.318316 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049bf01-1b86-4bca-b494-a4b41562ca07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.318981 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049bf01-1b86-4bca-b494-a4b41562ca07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:18:23.319279 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049bf01-1b86-4bca-b494-a4b41562ca07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:23.321396 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049bf01-1b86-4bca-b494-a4b41562ca07 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:23.329829 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:18:23.333359 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1429/2853] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:18:23.338258 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68d171-cc0f-4993-8ea5-dbc8f574d1b1 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:23.347815 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68d171-cc0f-4993-8ea5-dbc8f574d1b1 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:23.352734 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca324a-59eb-4e71-8759-04a5e36ab489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:23.353315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca324a-59eb-4e71-8759-04a5e36ab489 admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.353590 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca324a-59eb-4e71-8759-04a5e36ab489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.353857 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca324a-59eb-4e71-8759-04a5e36ab489 admin admin] PATH_INFO: `/v3/projects` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.354495 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca324a-59eb-4e71-8759-04a5e36ab489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:23.356451 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca324a-59eb-4e71-8759-04a5e36ab489 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:23.447993 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1425/2854] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:18:23 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 18:18:23.452105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049bf01-1b86-4bca-b494-a4b41562ca07 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:23.462522 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049bf01-1b86-4bca-b494-a4b41562ca07 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:23.517733 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23e30c-9520-49b2-8474-ea2df39d939d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:23.518348 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23e30c-9520-49b2-8474-ea2df39d939d admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.518835 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23e30c-9520-49b2-8474-ea2df39d939d admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.519112 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23e30c-9520-49b2-8474-ea2df39d939d admin admin] PATH_INFO: `/v3/users` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.519725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23e30c-9520-49b2-8474-ea2df39d939d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:23.521656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23e30c-9520-49b2-8474-ea2df39d939d admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:23.583646 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1430/2855] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:18:23 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:18:23.587819 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca324a-59eb-4e71-8759-04a5e36ab489 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:23.598370 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca324a-59eb-4e71-8759-04a5e36ab489 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:23.602959 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763fbff-5489-4384-a6ad-c91e3b32948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:23.603545 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763fbff-5489-4384-a6ad-c91e3b32948b admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.603823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763fbff-5489-4384-a6ad-c91e3b32948b admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.604089 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763fbff-5489-4384-a6ad-c91e3b32948b admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.604786 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763fbff-5489-4384-a6ad-c91e3b32948b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:23.606406 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763fbff-5489-4384-a6ad-c91e3b32948b admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:23.616056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1426/2856] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:18:23.620363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23e30c-9520-49b2-8474-ea2df39d939d admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:23.630256 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23e30c-9520-49b2-8474-ea2df39d939d admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:23.635439 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cbea9-a5cc-4afd-aca5-0238fee060d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:23.636006 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cbea9-a5cc-4afd-aca5-0238fee060d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.636276 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cbea9-a5cc-4afd-aca5-0238fee060d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.636591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cbea9-a5cc-4afd-aca5-0238fee060d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.637209 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cbea9-a5cc-4afd-aca5-0238fee060d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:23.638828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cbea9-a5cc-4afd-aca5-0238fee060d3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:23.649059 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1431/2857] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:18:23.652924 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763fbff-5489-4384-a6ad-c91e3b32948b admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:23.662871 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763fbff-5489-4384-a6ad-c91e3b32948b admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:23.667504 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82c3da-3565-437f-a243-bfefa0b0f3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:23.668128 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82c3da-3565-437f-a243-bfefa0b0f3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.668411 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82c3da-3565-437f-a243-bfefa0b0f3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.668724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82c3da-3565-437f-a243-bfefa0b0f3d6 admin admin] PATH_INFO: `/v3/projects/e50caa8d3d2f4e5d88a3f7f68d76115d/users/d8196e2fad744bcda762ceef50d13faa/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.684377 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82c3da-3565-437f-a243-bfefa0b0f3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50caa8d3d2f4e5d88a3f7f68d76115d, user_id=d8196e2fad744bcda762ceef50d13faa, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:23.687042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82c3da-3565-437f-a243-bfefa0b0f3d6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:23.737923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1427/2858] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:18:23 2022] PUT /identity/v3/projects/e50caa8d3d2f4e5d88a3f7f68d76115d/users/d8196e2fad744bcda762ceef50d13faa/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:18:23.808743 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127023c-5fa3-423b-b3ca-55c822beca60 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.809021 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127023c-5fa3-423b-b3ca-55c822beca60 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.809284 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127023c-5fa3-423b-b3ca-55c822beca60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.837050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-7127023c-5fa3-423b-b3ca-55c822beca60 None None] MFA Rules not processed for user `d8196e2fad744bcda762ceef50d13faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:18:23.838186 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7127023c-5fa3-423b-b3ca-55c822beca60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:18:23.905050 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1432/2859] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:18:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:18:23.916671 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82c3da-3565-437f-a243-bfefa0b0f3d6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:23.918397 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-1d82c3da-3565-437f-a243-bfefa0b0f3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:18:23.932856 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:18:23.975307 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82c3da-3565-437f-a243-bfefa0b0f3d6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:23.979452 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8d08f-b23d-4df5-92c1-b775dcfe4850 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e95d295337b4472be946d252ac3bd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:23.980098 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8d08f-b23d-4df5-92c1-b775dcfe4850 service glance] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:23.980372 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8d08f-b23d-4df5-92c1-b775dcfe4850 service glance] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:23.980673 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8d08f-b23d-4df5-92c1-b775dcfe4850 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:23.989252 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8d08f-b23d-4df5-92c1-b775dcfe4850 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:23.990712 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8d08f-b23d-4df5-92c1-b775dcfe4850 service glance] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:24.000045 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1428/2860] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:18:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:18:33.901771 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:18:34.006484 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=20 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:18:52.455693 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127023c-5fa3-423b-b3ca-55c822beca60 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:18:52.457163 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7127023c-5fa3-423b-b3ca-55c822beca60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:18:52.516591 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127023c-5fa3-423b-b3ca-55c822beca60 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:18:52.520766 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7372aa-72d6-4922-909f-66205a5a2241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:18:52.521358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7372aa-72d6-4922-909f-66205a5a2241 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:18:52.521629 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7372aa-72d6-4922-909f-66205a5a2241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:18:52.521895 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7372aa-72d6-4922-909f-66205a5a2241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:18:52.523601 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-7a7372aa-72d6-4922-909f-66205a5a2241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:18:52.603017 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7372aa-72d6-4922-909f-66205a5a2241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:18:52.604452 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7372aa-72d6-4922-909f-66205a5a2241 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:18:52.624777 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1433/2861] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:18:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:19:02.622155 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=63 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:07.291048 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b843393-0d40-4c9e-92e8-ed4676667df7 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:07.291558 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b843393-0d40-4c9e-92e8-ed4676667df7 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:07.291678 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b843393-0d40-4c9e-92e8-ed4676667df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:07.321695 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-2b843393-0d40-4c9e-92e8-ed4676667df7 None None] MFA Rules not processed for user `6ae6453ca9ec4a4eb3ec2e25900810cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:19:07.322887 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-2b843393-0d40-4c9e-92e8-ed4676667df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:07.385726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1429/2862] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:19:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:19:08.082239 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7372aa-72d6-4922-909f-66205a5a2241 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:08.091497 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7372aa-72d6-4922-909f-66205a5a2241 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:08.096090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54066ad3-40c4-4375-a976-2e42d36477e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:08.096090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54066ad3-40c4-4375-a976-2e42d36477e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:08.096090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54066ad3-40c4-4375-a976-2e42d36477e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:08.096995 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54066ad3-40c4-4375-a976-2e42d36477e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:08.098761 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-54066ad3-40c4-4375-a976-2e42d36477e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:08.113016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54066ad3-40c4-4375-a976-2e42d36477e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:08.114492 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54066ad3-40c4-4375-a976-2e42d36477e9 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:08.126516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1434/2863] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:19:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:19:08.577669 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b843393-0d40-4c9e-92e8-ed4676667df7 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:08.640753 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b843393-0d40-4c9e-92e8-ed4676667df7 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:08.645277 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:08.645889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:08.646164 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:08.646433 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] PATH_INFO: `/v3/users/d8196e2fad744bcda762ceef50d13faa` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:08.647570 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8196e2fad744bcda762ceef50d13faa)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:08.649513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:08.866679 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8196e2fad744bcda762ceef50d13faa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:08.937888 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8196e2fad744bcda762ceef50d13faa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:08.945520 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8196e2fad744bcda762ceef50d13faa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:08.946165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8196e2fad744bcda762ceef50d13faa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:08.946772 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8196e2fad744bcda762ceef50d13faa'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:08.964069 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1430/2864] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:19:08 2022] DELETE /identity/v3/users/d8196e2fad744bcda762ceef50d13faa => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:19:08.968528 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557b1ff-1722-4b61-8a5b-0fe74fd45795 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:08.968814 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557b1ff-1722-4b61-8a5b-0fe74fd45795 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:08.969092 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557b1ff-1722-4b61-8a5b-0fe74fd45795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:08.989227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-0557b1ff-1722-4b61-8a5b-0fe74fd45795 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:19:08.990458 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0557b1ff-1722-4b61-8a5b-0fe74fd45795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:09.048026 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1435/2865] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:19:09.058574 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:09.059976 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:09.087997 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:09.117360 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fd254-b2fe-4ed5-8be4-d973573da1a6 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:09.121016 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6958d-15f0-4a84-9995-3e8e16ef2344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:09.121987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6958d-15f0-4a84-9995-3e8e16ef2344 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:09.121987 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6958d-15f0-4a84-9995-3e8e16ef2344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:09.122347 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6958d-15f0-4a84-9995-3e8e16ef2344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:09.131006 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6958d-15f0-4a84-9995-3e8e16ef2344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:09.133407 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6958d-15f0-4a84-9995-3e8e16ef2344 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:09.142492 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1431/2866] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:19:09.539369 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557b1ff-1722-4b61-8a5b-0fe74fd45795 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:09.540864 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-0557b1ff-1722-4b61-8a5b-0fe74fd45795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:09.557146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557b1ff-1722-4b61-8a5b-0fe74fd45795 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:09.560710 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:09.571462 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:09.571807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:09.572090 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] PATH_INFO: `/v3/projects/e50caa8d3d2f4e5d88a3f7f68d76115d` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:09.573619 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50caa8d3d2f4e5d88a3f7f68d76115d)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:09.576336 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:09.679061 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50caa8d3d2f4e5d88a3f7f68d76115d', 'request_id': 'req-2f90119b-b891-4454-ad49-f6734ff81580'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:09.684531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50caa8d3d2f4e5d88a3f7f68d76115d', 'request_id': 'req-2f90119b-b891-4454-ad49-f6734ff81580'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:09.743104 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50caa8d3d2f4e5d88a3f7f68d76115d', 'request_id': 'req-2f90119b-b891-4454-ad49-f6734ff81580'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:09.755373 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] The token cache is being invalidate because project e50caa8d3d2f4e5d88a3f7f68d76115d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:19:09.755661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:09.756293 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-2f90119b-b891-4454-ad49-f6734ff81580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:09.757795 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1436/2867] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:19:09 2022] DELETE /identity/v3/projects/e50caa8d3d2f4e5d88a3f7f68d76115d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:19:09.762707 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65194d13-eca2-4df0-aead-947a9ace0b57 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:09.763208 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65194d13-eca2-4df0-aead-947a9ace0b57 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:09.763315 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65194d13-eca2-4df0-aead-947a9ace0b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:09.783506 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-65194d13-eca2-4df0-aead-947a9ace0b57 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:19:09.784635 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-65194d13-eca2-4df0-aead-947a9ace0b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:09.851650 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1432/2868] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 18:19:09.859772 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf85d-32a5-4426-9a48-0e9fec013305 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:09.860055 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf85d-32a5-4426-9a48-0e9fec013305 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:09.860329 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf85d-32a5-4426-9a48-0e9fec013305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:09.881762 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-e17cf85d-32a5-4426-9a48-0e9fec013305 None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:19:09.882970 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-e17cf85d-32a5-4426-9a48-0e9fec013305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:09.891066 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1437/2869] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:19:09.896139 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65194d13-eca2-4df0-aead-947a9ace0b57 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:09.905757 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65194d13-eca2-4df0-aead-947a9ace0b57 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:09.910313 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963066d0-5e87-4a45-84ec-5076021bdc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:09.910892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963066d0-5e87-4a45-84ec-5076021bdc66 admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:09.911163 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963066d0-5e87-4a45-84ec-5076021bdc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:09.911425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963066d0-5e87-4a45-84ec-5076021bdc66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:09.912056 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963066d0-5e87-4a45-84ec-5076021bdc66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80401) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 18:19:09.912348 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963066d0-5e87-4a45-84ec-5076021bdc66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:09.913943 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963066d0-5e87-4a45-84ec-5076021bdc66 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:09.925104 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1433/2870] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 18:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 18:19:09.929358 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cf85d-32a5-4426-9a48-0e9fec013305 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:09.938656 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cf85d-32a5-4426-9a48-0e9fec013305 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:09.943987 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a8eaa-bfd9-45e1-8fb5-c63f3030ec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:09.944602 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8eaa-bfd9-45e1-8fb5-c63f3030ec07 admin admin] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:09.944881 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8eaa-bfd9-45e1-8fb5-c63f3030ec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:09.945149 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8eaa-bfd9-45e1-8fb5-c63f3030ec07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:09.945805 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a8eaa-bfd9-45e1-8fb5-c63f3030ec07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:09.947768 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a8eaa-bfd9-45e1-8fb5-c63f3030ec07 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:10.035909 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1438/2871] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 18:19:09 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:19:10.040027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963066d0-5e87-4a45-84ec-5076021bdc66 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:10.050717 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963066d0-5e87-4a45-84ec-5076021bdc66 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:10.106362 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15f68-05ed-44c5-9ded-dab970c328de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:10.107027 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15f68-05ed-44c5-9ded-dab970c328de admin admin] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:10.107296 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15f68-05ed-44c5-9ded-dab970c328de admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:10.107830 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15f68-05ed-44c5-9ded-dab970c328de admin admin] PATH_INFO: `/v3/users` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:10.108193 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15f68-05ed-44c5-9ded-dab970c328de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:10.110273 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15f68-05ed-44c5-9ded-dab970c328de admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:10.213669 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1434/2872] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 18:19:10 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 18:19:10.218144 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a8eaa-bfd9-45e1-8fb5-c63f3030ec07 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:10.228581 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a8eaa-bfd9-45e1-8fb5-c63f3030ec07 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:10.233241 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aecf7-e9cb-4b4d-9f65-19ea95280aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:10.233850 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aecf7-e9cb-4b4d-9f65-19ea95280aee admin admin] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:10.234123 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aecf7-e9cb-4b4d-9f65-19ea95280aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:10.234387 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aecf7-e9cb-4b4d-9f65-19ea95280aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:10.235016 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aecf7-e9cb-4b4d-9f65-19ea95280aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:10.237160 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aecf7-e9cb-4b4d-9f65-19ea95280aee admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:10.248873 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1439/2873] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 18:19:10.253521 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15f68-05ed-44c5-9ded-dab970c328de admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:10.263923 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15f68-05ed-44c5-9ded-dab970c328de admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:10.269121 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338415c-b323-417f-827e-501f652732bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:10.270975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338415c-b323-417f-827e-501f652732bd admin admin] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:10.270975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338415c-b323-417f-827e-501f652732bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:10.270975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338415c-b323-417f-827e-501f652732bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:10.271281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338415c-b323-417f-827e-501f652732bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:10.272694 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338415c-b323-417f-827e-501f652732bd admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:10.287677 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1435/2874] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 18:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 18:19:10.289150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aecf7-e9cb-4b4d-9f65-19ea95280aee admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:10.305401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aecf7-e9cb-4b4d-9f65-19ea95280aee admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:10.312824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046ba13-0170-4252-973d-9208f099cbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:10.313842 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ba13-0170-4252-973d-9208f099cbb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:10.314294 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ba13-0170-4252-973d-9208f099cbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:10.314745 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ba13-0170-4252-973d-9208f099cbb7 admin admin] PATH_INFO: `/v3/projects/30f1acedcee44ba898d033e1f40cc633/users/6da26d47517244b6a59d6dd8b2c2c552/roles/f61a519ee1a74ddeb7cb809ee43d9825` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:10.332661 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046ba13-0170-4252-973d-9208f099cbb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f1acedcee44ba898d033e1f40cc633, user_id=6da26d47517244b6a59d6dd8b2c2c552, role_id=f61a519ee1a74ddeb7cb809ee43d9825)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:10.336140 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046ba13-0170-4252-973d-9208f099cbb7 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:10.435704 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:10.441282 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1440/2875] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 18:19:10 2022] PUT /identity/v3/projects/30f1acedcee44ba898d033e1f40cc633/users/6da26d47517244b6a59d6dd8b2c2c552/roles/f61a519ee1a74ddeb7cb809ee43d9825 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 18:19:10.452633 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934daa2e-8b9c-4e5c-b20d-58d41fa59e48 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:10.452914 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934daa2e-8b9c-4e5c-b20d-58d41fa59e48 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:10.453182 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934daa2e-8b9c-4e5c-b20d-58d41fa59e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:10.480983 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-934daa2e-8b9c-4e5c-b20d-58d41fa59e48 None None] MFA Rules not processed for user `6da26d47517244b6a59d6dd8b2c2c552`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:19:10.482117 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-934daa2e-8b9c-4e5c-b20d-58d41fa59e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:10.503670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:10.551458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1436/2876] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:19:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:19:10.562328 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046ba13-0170-4252-973d-9208f099cbb7 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:10.564041 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-a046ba13-0170-4252-973d-9208f099cbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:10.620332 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046ba13-0170-4252-973d-9208f099cbb7 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:10.623964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee495be5-24de-4c15-b1e7-242159808fa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:10.624586 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee495be5-24de-4c15-b1e7-242159808fa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:10.630444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee495be5-24de-4c15-b1e7-242159808fa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:10.630444 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee495be5-24de-4c15-b1e7-242159808fa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:10.633605 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee495be5-24de-4c15-b1e7-242159808fa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:10.635053 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee495be5-24de-4c15-b1e7-242159808fa5 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:10.643905 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1441/2877] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:19:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 18:19:20.737021 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=17 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:20.787634 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=27 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:24.991670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934daa2e-8b9c-4e5c-b20d-58d41fa59e48 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:24.991670 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-934daa2e-8b9c-4e5c-b20d-58d41fa59e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:25.043247 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934daa2e-8b9c-4e5c-b20d-58d41fa59e48 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:25.047061 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:25.047660 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:25.047938 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:25.048205 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:25.050511 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:25.061227 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:25.062584 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:25.071130 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1437/2878] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:19:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:19:27.557640 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187c537-99d3-4de7-822c-581eaf20d047 None None] REQUEST_METHOD: `POST` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:27.558121 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187c537-99d3-4de7-822c-581eaf20d047 None None] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:27.558216 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187c537-99d3-4de7-822c-581eaf20d047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:27.576742 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.auth.core [None req-3187c537-99d3-4de7-822c-581eaf20d047 None None] MFA Rules not processed for user `0c911d938fd64f25844903809d56b263`. Rule list: `[]` (Enabled: `True`). {{(pid=80402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:19:27.577964 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-3187c537-99d3-4de7-822c-581eaf20d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:27.633012 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1442/2879] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:19:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:19:28.687037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:28.688622 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:28.766057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7860b-d565-4abf-b9bf-f9b5480e9c50 service neutron] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:28.771601 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc9be4-863d-4ac1-9c27-c47d1fc8b141 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:28.772200 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc9be4-863d-4ac1-9c27-c47d1fc8b141 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:28.772478 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc9be4-863d-4ac1-9c27-c47d1fc8b141 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:28.772774 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc9be4-863d-4ac1-9c27-c47d1fc8b141 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:28.774509 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-c7bc9be4-863d-4ac1-9c27-c47d1fc8b141 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:28.786916 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc9be4-863d-4ac1-9c27-c47d1fc8b141 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:28.788517 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc9be4-863d-4ac1-9c27-c47d1fc8b141 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:28.799018 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1438/2880] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:19:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:19:31.655398 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187c537-99d3-4de7-822c-581eaf20d047 None None] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:31.723565 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187c537-99d3-4de7-822c-581eaf20d047 None None] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:31.728460 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:31.729139 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:31.729414 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:31.729685 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] PATH_INFO: `/v3/users/6da26d47517244b6a59d6dd8b2c2c552` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:31.730862 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da26d47517244b6a59d6dd8b2c2c552)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:31.732942 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:31.894035 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da26d47517244b6a59d6dd8b2c2c552'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:31.960381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da26d47517244b6a59d6dd8b2c2c552'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:31.967988 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da26d47517244b6a59d6dd8b2c2c552'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:31.968665 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da26d47517244b6a59d6dd8b2c2c552'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:31.971295 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da26d47517244b6a59d6dd8b2c2c552'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:31.994731 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1443/2881] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:19:31 2022] DELETE /identity/v3/users/6da26d47517244b6a59d6dd8b2c2c552 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:19:31.997294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706da49-37e8-4b08-b9eb-02f3fed8668e None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:31.997796 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706da49-37e8-4b08-b9eb-02f3fed8668e None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:32.004829 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706da49-37e8-4b08-b9eb-02f3fed8668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:32.020766 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-6706da49-37e8-4b08-b9eb-02f3fed8668e None None] MFA Rules not processed for user `2c322d3415e942c2b44d2bbf25f886a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:19:32.022115 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6706da49-37e8-4b08-b9eb-02f3fed8668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:32.088949 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1439/2882] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:19:32.100725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:32.102350 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:32.140923 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:32.171516 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838ef11-1f86-49f4-963a-3d308caff66c admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:32.175258 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7cfba-a726-4126-a8a9-55d362917e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:32.176531 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7cfba-a726-4126-a8a9-55d362917e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:32.176807 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7cfba-a726-4126-a8a9-55d362917e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:32.177105 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7cfba-a726-4126-a8a9-55d362917e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:32.187613 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7cfba-a726-4126-a8a9-55d362917e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:32.189091 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7cfba-a726-4126-a8a9-55d362917e71 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:32.201008 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1444/2883] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:19:32.518070 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706da49-37e8-4b08-b9eb-02f3fed8668e None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:32.520294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-6706da49-37e8-4b08-b9eb-02f3fed8668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:32.555988 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=100 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:32.595499 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706da49-37e8-4b08-b9eb-02f3fed8668e None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:32.601674 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:32.602281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:32.602548 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:32.602814 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:32.604580 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:32.617566 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:32.619142 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:32.630657 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1440/2884] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:19:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:19:32.682068 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7cfba-a726-4126-a8a9-55d362917e71 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:32.683604 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-ccd7cfba-a726-4126-a8a9-55d362917e71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:32.697160 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7cfba-a726-4126-a8a9-55d362917e71 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:32.702203 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:32.703023 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:32.703538 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:32.703811 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] PATH_INFO: `/v3/projects/30f1acedcee44ba898d033e1f40cc633` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:32.705150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f1acedcee44ba898d033e1f40cc633)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:32.707580 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:32.798178 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f1acedcee44ba898d033e1f40cc633', 'request_id': 'req-d60538ee-53db-4658-a574-a8077a0a39d8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:32.805914 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f1acedcee44ba898d033e1f40cc633', 'request_id': 'req-d60538ee-53db-4658-a574-a8077a0a39d8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:32.874828 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f1acedcee44ba898d033e1f40cc633', 'request_id': 'req-d60538ee-53db-4658-a574-a8077a0a39d8'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:32.887307 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] The token cache is being invalidate because project 30f1acedcee44ba898d033e1f40cc633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:19:32.887592 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:32.888227 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:19:32.889817 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1445/2885] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:19:32 2022] DELETE /identity/v3/projects/30f1acedcee44ba898d033e1f40cc633 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:19:42.755649 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=19 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:42.884077 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:19:49.340477 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:19:49.342017 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:49.356889 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3bb11-d025-4c64-b7a4-b87c7d0af7de service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:19:49.421342 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e7b4-4975-4696-8bbb-411055dbafb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c496da229db34f90a4b3fdcfa582dca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:19:49.421946 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e7b4-4975-4696-8bbb-411055dbafb6 service nova] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:19:49.422220 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e7b4-4975-4696-8bbb-411055dbafb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:19:49.422493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34e7b4-4975-4696-8bbb-411055dbafb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:19:49.424241 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-fa34e7b4-4975-4696-8bbb-411055dbafb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:19:49.437374 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34e7b4-4975-4696-8bbb-411055dbafb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:19:49.438975 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34e7b4-4975-4696-8bbb-411055dbafb6 service nova] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:19:49.450172 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1441/2886] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:19:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:19:59.445350 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=34 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:20:21.020784 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:20:21.025325 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:20:21.087938 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60538ee-53db-4658-a574-a8077a0a39d8 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:20:21.092482 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:20:21.093189 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:20:21.093467 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:20:21.093738 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] PATH_INFO: `/v3/users/6ae6453ca9ec4a4eb3ec2e25900810cc` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:20:21.094853 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae6453ca9ec4a4eb3ec2e25900810cc)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:20:21.096798 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:20:21.259288 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae6453ca9ec4a4eb3ec2e25900810cc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.325245 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae6453ca9ec4a4eb3ec2e25900810cc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.332292 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae6453ca9ec4a4eb3ec2e25900810cc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.332946 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae6453ca9ec4a4eb3ec2e25900810cc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.333557 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae6453ca9ec4a4eb3ec2e25900810cc'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.350968 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1446/2887] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:20:21 2022] DELETE /identity/v3/users/6ae6453ca9ec4a4eb3ec2e25900810cc => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:20:21.595895 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e7b4-4975-4696-8bbb-411055dbafb6 service nova] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:20:21.607036 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34e7b4-4975-4696-8bbb-411055dbafb6 service nova] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:20:21.664608 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:20:21.665231 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:20:21.665510 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:20:21.665781 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] PATH_INFO: `/v3/projects/a7a885b9c1574a8eb0a62b5253c9f8c2` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:20:21.666892 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a885b9c1574a8eb0a62b5253c9f8c2)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:20:21.668823 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:20:21.788398 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a885b9c1574a8eb0a62b5253c9f8c2', 'request_id': 'req-d36b47da-522d-4ba3-9476-433557c15f01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.794186 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a885b9c1574a8eb0a62b5253c9f8c2', 'request_id': 'req-d36b47da-522d-4ba3-9476-433557c15f01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.853082 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a885b9c1574a8eb0a62b5253c9f8c2', 'request_id': 'req-d36b47da-522d-4ba3-9476-433557c15f01'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.864560 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] The token cache is being invalidate because project a7a885b9c1574a8eb0a62b5253c9f8c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80401) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:20:21.864852 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.865493 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-d36b47da-522d-4ba3-9476-433557c15f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:20:21.867457 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1442/2888] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:20:21 2022] DELETE /identity/v3/projects/a7a885b9c1574a8eb0a62b5253c9f8c2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:20:31.347440 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:20:31.861629 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:21:34.746818 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:21:34.750056 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:21:34.807143 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a4ae-d110-4412-ad6c-f5f6353b37e5 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:21:34.855120 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4aef937fb047f6869a156a68e7bf2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:21:34.855712 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:21:34.856100 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:21:34.856396 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:21:34.858159 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:21:34.914459 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:21:34.915876 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:21:34.924716 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1447/2889] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:21:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:21:43.341742 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893af594-f76c-4fd3-8df1-3b122f870514 None None] REQUEST_METHOD: `POST` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:21:43.345039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893af594-f76c-4fd3-8df1-3b122f870514 None None] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:21:43.345039 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893af594-f76c-4fd3-8df1-3b122f870514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:21:43.361224 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.auth.core [None req-893af594-f76c-4fd3-8df1-3b122f870514 None None] MFA Rules not processed for user `0c911d938fd64f25844903809d56b263`. Rule list: `[]` (Enabled: `True`). {{(pid=80401) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 18:21:43.362400 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-893af594-f76c-4fd3-8df1-3b122f870514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:21:43.369487 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1443/2890] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 18:21:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 18:21:43.683821 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:21:43.685150 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:21:43.741908 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8709e66-16de-4508-8b9a-0d4447e02502 service cinder] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:21:43.745515 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:21:43.746108 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:21:43.746381 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:21:43.746647 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:21:43.748372 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:21:43.758498 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:21:43.759898 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:21:43.767832 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1448/2891] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:21:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:21:53.356382 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=3 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:21:53.764167 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=94 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:23:13.849523 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893af594-f76c-4fd3-8df1-3b122f870514 None None] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:23:13.851011 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-893af594-f76c-4fd3-8df1-3b122f870514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:13.918974 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893af594-f76c-4fd3-8df1-3b122f870514 None None] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:23:13.924052 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:23:13.924746 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:23:13.925020 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:23:13.925281 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:23:13.926972 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:13.939040 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:23:13.940699 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:23:13.950425 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1444/2892] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:23:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:23:23.946219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=32 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:23:47.641721 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:23:47.643146 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:47.654225 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88838eb3-0f8b-45a6-9a4c-14f66e586778 service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:23:47.657884 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262f150-d2af-4dfd-a077-71b1eae58a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f02466042946ee96e37c5adc27b01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:23:47.658422 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f150-d2af-4dfd-a077-71b1eae58a2a service neutron] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:23:47.663841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f150-d2af-4dfd-a077-71b1eae58a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:23:47.663841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f150-d2af-4dfd-a077-71b1eae58a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:23:47.663841 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-9262f150-d2af-4dfd-a077-71b1eae58a2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:47.722816 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262f150-d2af-4dfd-a077-71b1eae58a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:23:47.724242 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262f150-d2af-4dfd-a077-71b1eae58a2a service neutron] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:23:47.734133 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1449/2893] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:23:51.158617 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:23:51.160014 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:51.173165 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bce10-6016-4afa-ae24-e162d2840bcb service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:23:51.179137 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:23:51.179754 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:23:51.180037 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:23:51.180294 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:23:51.182087 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:51.194672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:23:51.197033 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:23:51.207625 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1445/2894] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:23:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:23:54.543725 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262f150-d2af-4dfd-a077-71b1eae58a2a service neutron] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:23:54.559321 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262f150-d2af-4dfd-a077-71b1eae58a2a service neutron] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:23:54.565228 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac91cda-f77b-4940-a309-e9eaf0f53d5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:23:54.565824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac91cda-f77b-4940-a309-e9eaf0f53d5b service placement] REQUEST_METHOD: `GET` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:23:54.566119 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac91cda-f77b-4940-a309-e9eaf0f53d5b service placement] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:23:54.566363 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac91cda-f77b-4940-a309-e9eaf0f53d5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:23:54.568129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-1ac91cda-f77b-4940-a309-e9eaf0f53d5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:54.580401 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac91cda-f77b-4940-a309-e9eaf0f53d5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:23:54.582067 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac91cda-f77b-4940-a309-e9eaf0f53d5b service placement] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:23:54.592775 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1450/2895] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:23:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:23:54.685739 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:23:54.687427 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:54.702780 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bbb16-06e4-4c9b-9e4e-d50c2796775e service placement] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:23:54.707642 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:23:54.709651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:23:54.709651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:23:54.709651 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] PATH_INFO: `/v3/users/0c911d938fd64f25844903809d56b263` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:23:54.710008 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c911d938fd64f25844903809d56b263)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:23:54.712246 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:23:54.902936 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c911d938fd64f25844903809d56b263'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:54.983816 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c911d938fd64f25844903809d56b263'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:54.991880 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c911d938fd64f25844903809d56b263'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:54.992639 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c911d938fd64f25844903809d56b263'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:54.993331 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c911d938fd64f25844903809d56b263'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.011057 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1446/2896] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:23:54 2022] DELETE /identity/v3/users/0c911d938fd64f25844903809d56b263 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:23:55.252542 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac91cda-f77b-4940-a309-e9eaf0f53d5b service placement] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:23:55.265385 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac91cda-f77b-4940-a309-e9eaf0f53d5b service placement] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:23:55.325776 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:23:55.326399 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:23:55.328129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:23:55.328129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] PATH_INFO: `/v3/projects/c98de54846de4169b0ea5cca7aae3cf0` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:23:55.328129 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c98de54846de4169b0ea5cca7aae3cf0)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:23:55.330168 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:23:55.429455 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c98de54846de4169b0ea5cca7aae3cf0', 'request_id': 'req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.435032 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c98de54846de4169b0ea5cca7aae3cf0', 'request_id': 'req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.499971 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c98de54846de4169b0ea5cca7aae3cf0', 'request_id': 'req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.512107 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] The token cache is being invalidate because project c98de54846de4169b0ea5cca7aae3cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:23:55.512394 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.513054 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.514590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1451/2897] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:23:55 2022] DELETE /identity/v3/projects/c98de54846de4169b0ea5cca7aae3cf0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:23:55.518724 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:23:55.520114 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:55.533214 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438aac14-19a0-45c5-b6e9-7a72bc40845e admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:23:55.588905 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:23:55.589450 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:23:55.589726 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:23:55.589995 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] PATH_INFO: `/v3/users/07fa6d8acbb44b5abac88a7f63fee601` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:23:55.591124 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07fa6d8acbb44b5abac88a7f63fee601)` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:23:55.593064 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:23:55.791276 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07fa6d8acbb44b5abac88a7f63fee601'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.857758 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07fa6d8acbb44b5abac88a7f63fee601'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.863181 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:23:55.865844 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07fa6d8acbb44b5abac88a7f63fee601'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.866464 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07fa6d8acbb44b5abac88a7f63fee601'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.867060 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.notifications [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07fa6d8acbb44b5abac88a7f63fee601'} {{(pid=80401) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:55.883513 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1447/2898] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 18:23:55 2022] DELETE /identity/v3/users/07fa6d8acbb44b5abac88a7f63fee601 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:23:56.093953 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] Authenticating user token {{(pid=80402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:23:56.095508 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.fernet_utils [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:23:56.110618 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4900fc-16bd-4cc5-bda0-802ee9c0dcd3 admin admin] Validating token access rules against request {{(pid=80402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:23:56.120270 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:23:56.170222 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c322d3415e942c2b44d2bbf25f886a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c1d22d0f80a4861a903f2c834f48928', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:23:56.170848 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:23:56.171153 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:23:56.171421 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] PATH_INFO: `/v3/projects/01c1922dfd9e47afb271fbfdbc4f0663` {{(pid=80402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:23:56.172590 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c1922dfd9e47afb271fbfdbc4f0663)` {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:23:56.174589 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] RBAC: Authorization granted {{(pid=80402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:23:56.297824 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c1922dfd9e47afb271fbfdbc4f0663', 'request_id': 'req-6db2563f-3348-45d1-813d-91e67238a836'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:56.304064 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c1922dfd9e47afb271fbfdbc4f0663', 'request_id': 'req-6db2563f-3348-45d1-813d-91e67238a836'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:56.378083 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c1922dfd9e47afb271fbfdbc4f0663', 'request_id': 'req-6db2563f-3348-45d1-813d-91e67238a836'} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:56.390118 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] The token cache is being invalidate because project 01c1922dfd9e47afb271fbfdbc4f0663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 18:23:56.390409 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:56.391087 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG keystone.notifications [None req-6db2563f-3348-45d1-813d-91e67238a836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 18:23:56.392727 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: [pid: 80402|app: 0|req: 1452/2899] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 18:23:56 2022] DELETE /identity/v3/projects/01c1922dfd9e47afb271fbfdbc4f0663 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 18:24:05.966230 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:24:06.386999 dv-cinder-852364-1-iscsi devstack@keystone.service[80402]: DEBUG dbcounter [-] [80402] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 18:28:20.835458 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] Authenticating user token {{(pid=80401) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 18:28:20.837219 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:28:20.975527 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b91210-9b79-46d2-9c80-527fe02f0c20 admin admin] Validating token access rules against request {{(pid=80401) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 18:28:21.032547 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888742e-9159-404a-95fd-263de3b4dbb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b81bdfeedcba4b0c93f13d14318b2151', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35b6705a8d9647d3a9ea2b66510f8add', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80401) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 18:28:21.033126 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888742e-9159-404a-95fd-263de3b4dbb9 service placement] REQUEST_METHOD: `GET` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 18:28:21.033404 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888742e-9159-404a-95fd-263de3b4dbb9 service placement] SCRIPT_NAME: `/identity` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 18:28:21.033672 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888742e-9159-404a-95fd-263de3b4dbb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80401) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 18:28:21.035391 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.fernet_utils [None req-7888742e-9159-404a-95fd-263de3b4dbb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80401) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 18:28:21.047042 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888742e-9159-404a-95fd-263de3b4dbb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 18:28:21.048641 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888742e-9159-404a-95fd-263de3b4dbb9 service placement] RBAC: Authorization granted {{(pid=80401) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 18:28:21.058460 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: [pid: 80401|app: 0|req: 1448/2900] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 18:28:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 18:28:31.054242 dv-cinder-852364-1-iscsi devstack@keystone.service[80401]: DEBUG dbcounter [-] [80401] Writing DB stats keystone:SELECT=89 {{(pid=80401) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}